-- Logs begin at Wed 2022-06-29 07:48:24 UTC, end at Wed 2022-06-29 09:04:05 UTC. -- Jun 29 08:21:08.232865 dv-cinder-797608-37-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 29 08:21:08.240539 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 29 08:21:08 2022] *** Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: nodename: dv-cinder-797608-37-iscsi Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: machine: x86_64 Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: clock source: unix Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: pcre jit disabled Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: detected number of CPU cores: 4 Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: current working directory: / Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: detected binary path: /usr/bin/uwsgi-core Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: your processes number limit is 47680 Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: your memory page size is 4096 bytes Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: detected max file descriptor number: 2048 Jun 29 08:21:08.242575 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: lock engine: pthread robust mutexes Jun 29 08:21:08.243358 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: thunder lock: enabled Jun 29 08:21:08.243636 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 29 08:21:08.243720 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 29 08:21:08.286617 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: Python main interpreter initialized at 0x55d45bb2f9c0 Jun 29 08:21:08.286617 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: python threads support enabled Jun 29 08:21:08.286617 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: your server socket listen backlog is limited to 100 connections Jun 29 08:21:08.286617 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: your mercy for graceful operations on workers is 90 seconds Jun 29 08:21:08.287041 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: mapped 403077 bytes (393 KB) for 2 cores Jun 29 08:21:08.287041 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: *** Operational MODE: preforking *** Jun 29 08:21:08.287348 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: *** uWSGI is running in multiple interpreter mode *** Jun 29 08:21:08.287348 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: spawned uWSGI master process (pid: 80119) Jun 29 08:21:08.287575 dv-cinder-797608-37-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 29 08:21:08.288633 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: spawned uWSGI worker 1 (pid: 80130, cores: 1) Jun 29 08:21:08.288633 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: spawned uWSGI worker 2 (pid: 80131, cores: 1) Jun 29 08:21:08.288633 dv-cinder-797608-37-iscsi devstack@keystone.service[80119]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 29 08:21:13.707912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 08:21:13.707912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: from cryptography.utils import int_from_bytes Jun 29 08:21:13.708792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 08:21:13.708792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: from cryptography.utils import int_from_bytes Jun 29 08:21:13.709212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 08:21:13.709212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: from cryptography.utils import int_from_bytes Jun 29 08:21:13.709321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 08:21:13.709321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: from cryptography.utils import int_from_bytes Jun 29 08:21:13.756864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 29 08:21:13.757034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 29 08:21:13.757107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 29 08:21:13.757198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] command line args: [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 29 08:21:13.757342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 29 08:21:13.757490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 29 08:21:13.757570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] command line args: [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 29 08:21:13.757647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ================================================================================ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 29 08:21:13.757806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 29 08:21:13.757892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] admin_token = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.757959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ================================================================================ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 29 08:21:13.758109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config_dir = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.758238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] admin_token = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.758319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.758386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_dir = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.758451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] config_source = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.758580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.758669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] debug = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.758774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] config_source = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.758835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.759102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] debug = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.759102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.759256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.759321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.759376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.759511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.759595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.759786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.759900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_config_append = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760270 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760445 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_config_append = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_dir = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760693 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_dir = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.760955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.761138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.761261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.761387 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.761475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.761590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.761673 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.761867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.762007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.762076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.762149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.762149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.762271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.762442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.762638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.762779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.762918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.763021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.763115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.763182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.763244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.763309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.763373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.763521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.763637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.763754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.763829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.764019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.764151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.764264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.764542 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.764703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] publish_errors = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.764805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.764922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] publish_errors = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.765024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.765151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.765236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.765314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.765382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.765545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.765667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.765761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.765839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.766014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] standard_threads = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.766135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.766227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.766353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] standard_threads = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.766429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.766498 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.766640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.766799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.766883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_journal = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.766969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.767046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_json = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.767117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_journal = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.767191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_stderr = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.767261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_json = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.767330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] use_syslog = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.767406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_stderr = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.767531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] use_syslog = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.767605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.767713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 08:21:13.767798 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.767994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.768116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.768195 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.768266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.768457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.768538 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.768691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.768772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.768931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.769017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.769165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.769244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.769398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.769482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.769630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.769837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.770028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.770109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.770271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.770393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.770544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.770633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.770901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.770989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.771144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.771225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.771392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.771471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.771650 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.771740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.771893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.771967 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.772116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.772202 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.772369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.772463 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.772612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.772691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.772850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.772933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.773085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.773159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.773335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.773427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.773691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.773771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.773936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.774016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.774171 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.774251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.774384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.774460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.774611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.774684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.774863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.774941 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.775098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.775175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.775343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.775426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.775574 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.775650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.775803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.775876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.776034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.776117 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.776279 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.776355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.776515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.776589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.776757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.776843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.776986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.777066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.777221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.777296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.777454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.777607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.777689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.777763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.777896 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.777979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.778127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.778211 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.778353 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.778427 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.778579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.778659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.778841 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.778931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.779071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.779147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.779309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.779384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.779540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.779650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.779768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.779867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.780002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.780114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.780344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.780474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.780474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.780648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.780759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.780840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.780940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.781028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.781131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.781214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.781367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.781442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.781600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.781687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.781824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.781899 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.782052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.782132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.782292 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.782375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.782523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.782639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.782780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.782864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.783013 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.783139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.783262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.783406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.783490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.783565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.783717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.783803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.783950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.784033 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.784186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.784277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.784434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.784513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.784675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.784760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.784914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.784990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.785150 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.785224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.785408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.785496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.785641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.785719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.785877 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.785960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.786108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.786185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.786345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.786423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.786579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.786655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.786838 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.786918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.787073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.787149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.787303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.787385 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.787533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.787617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.787765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.787848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.787994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.788071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.788250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.788327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.788484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.788561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.788717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.788793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.788939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.789018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.789174 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.789252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.789405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.789493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.789636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.789722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.789871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.789948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.790098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.790173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.790323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.790406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.790549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.790626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.790805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.790890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.791036 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.791114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.791266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.791343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.791495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.791570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.791732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.791808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.791963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.792041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.792216 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.792299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.792456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.792531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.792686 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.external = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.792765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.792914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.external = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.792989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.793143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.793219 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.793420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.793509 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.793649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.password = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.793729 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.793875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.password = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.793951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] auth.token = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.794104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] auth.token = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.794193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.794333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.794415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.794564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.794641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.794815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.794906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.795046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.795131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.795292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.795370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.795517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.795598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.795750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.795836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.795979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.796059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.796225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.796311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.796467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.796543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.796702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.796781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.796934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.797018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.797162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.797238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.797393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.797469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.797655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.797741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.797885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.797970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.798136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.798218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.798391 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.798481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.798650 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.798757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.798930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.799006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.799174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.799263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.799419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.799495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.799649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.799725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.799889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.799979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.800123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.800200 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.800363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.800439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.800591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.800676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.800820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.800896 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.801102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.801180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.801354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.801541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.801638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.801830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.801917 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.802067 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.802148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.802312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.802392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.802546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.802622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.802814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.802895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.803047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.803121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.803285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.803362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.803519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.803603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.803751 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.803835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.803979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.804063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.804216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.804305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.804446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.804522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.804684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.804760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.804912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.804996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.805140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.805218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.805369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.805454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.805604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.805688 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.805836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.805913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.806075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.806157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.806323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.806407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.806563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.806639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.806836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.806915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.807085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.807175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.807318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.807395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.807547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.807631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.807783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.807869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.808013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.808090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.808246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.808322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.808478 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.808569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.808711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.808787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.808956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.809064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.809179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.809278 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.809422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.809500 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.809650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.809727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.809890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.809974 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.810132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.810209 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.810366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.810443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.810589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.810674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.810849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.810928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.811075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.811152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.811308 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.811398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.811533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.811609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.811769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.811855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.812003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.812087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.812239 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.812316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.812474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.812549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.812712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.812798 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.812938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.813014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.813164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.813241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.813406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.813489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.813635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.813713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.813864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.813941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.814097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.814183 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.814330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.814408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.814558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.814635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.814813 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.814908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.815044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.815146 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.815282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.815379 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.815539 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.815624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.815768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.815845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.816014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.816091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.816265 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.816344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.816509 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.816763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.816763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.816898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.816978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.817051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.817202 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.817311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.817430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.817556 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.817676 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.817775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.817920 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.817997 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.818162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.818239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.818408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.818486 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.818652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.818780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.818911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.818989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.819156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.819237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.819395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.819472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.819635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.819720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.819869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.819947 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.820100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.820177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.820333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.820409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.820569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.820647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.820800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.820876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.821045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.821121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.821274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.821351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.821514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.821592 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.821741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.821818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.821971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.822048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.822205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.822281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.822438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.822519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.822674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.822815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.822934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.823012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.823169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.823246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.823397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.823475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.823631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.823709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.823869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.823946 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.824100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.824177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.824345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.824422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.824585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.824661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.824822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.824900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.825067 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.825143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.825298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.825372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.825532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.825610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.825764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.825841 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.825999 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.826075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.826226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.826340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.826473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.826551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.826728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.826808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.826971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.827051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.827213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.827296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.827457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.827534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.827703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.827782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.828020 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.828186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.828272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.828415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.828492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.828660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.828737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.828889 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.828972 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.829133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.829211 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.829374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.829451 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.829617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.829770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.829849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.830005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.830240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.830317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.830478 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.830563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.830742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.830822 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.830990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.831073 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.831220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.831309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.831472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.831548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.831704 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.831781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.831944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.832028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.832314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.832417 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.832535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.832718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.832810 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.833017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.833116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.833255 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.833331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.833489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.833584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.833722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.833799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.833957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.834039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.834187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.834272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.834420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.834498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.834656 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.834779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.834910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.834999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.835147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.835225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.835384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.835461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.835627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.835711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.835860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.835936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.836100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.836176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.836338 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.836424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.836576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.836653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.836808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.836884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.837043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.837129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.837277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.837354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.837511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.837586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.837748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.837833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.838318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.838399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.838561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.838639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.838834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.838924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.839066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.839144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.839312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.839398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.839598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.839675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.839837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.839923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.840077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.840164 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.840310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.840388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.840546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.840624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.840782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.840860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.841016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.841092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.841251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.841327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.841483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.841562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.841717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.841793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.841947 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.842023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.842185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.842263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.842417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.842492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.842649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.842743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.842902 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.842981 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.843143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.843219 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.843372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.843448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.843527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80130) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 29 08:21:13.843601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.843827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.844060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 08:21:13.844229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80131) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 29 08:21:13.873812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:13.874042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:13.874167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:13.874277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:13.874402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:13.874662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:13.882896 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:13.883146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:13.910513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:13.910784 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:13.911775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:13.912005 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.055084 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.055878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.103431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.103623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.104728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.104970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.167989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.168174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.168260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.168428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.168585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.168585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.168725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.168807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.168882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.169075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.169199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.169336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.194282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.194514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.194633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.194854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.194929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.194991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.195057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.195142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.195220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.195282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.195441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.195567 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.195639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.195734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.206652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.206954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.219703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.219938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.220135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.220253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.220390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.220482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.220630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.220719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.220840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.220961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.221080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80130) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.221212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.221212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.221443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80131) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 08:21:14.238268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.238500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 08:21:14.238890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.239074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.239219 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 08:21:14.239302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 08:21:14.239380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.239587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.239717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.239789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 08:21:14.239880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.240082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.240207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.240352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.240536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.240785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.250848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.251125 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.251387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.251513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.251680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.251866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.252059 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.252328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.259975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.260819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.263348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.263621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.263902 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.264082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.264212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.264378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.264503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.264654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.264779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.264931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.265066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.265165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.265253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.265451 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.265575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.265684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.265784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.265972 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.266098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.266204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.266496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.266761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.292582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.293425 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.294183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.294461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.295057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.295332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.299719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.299994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.300390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.300593 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.300850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.301297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.307773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.307997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.308584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.308861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.312462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.312749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.313026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.313308 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.313449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.313535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.313630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.313809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.313902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.314108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.314245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.314328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.314402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.314672 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.314957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.315224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.333185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.333455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.333996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.334269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.338683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.339177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.342328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.342619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.342811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.343117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.347780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.348441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.351394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.352179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.355052 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.355346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.355631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.355847 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.355945 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.356160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.356354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.356448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.356576 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.356711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.356977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.357253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.375507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.375788 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.376060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.376315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.376460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.376750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.377016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.377275 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.384367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.385181 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.386057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.386331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.386904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.387179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.389627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.389906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.390183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.390308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.390519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.390730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.390836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.390972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.391167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.391445 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.404150 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.404437 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.404713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.404990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.405113 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.405278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.405419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.405549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.405814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.406101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.406852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.407155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.407697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.407996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.425003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.426097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.428339 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.429269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.431683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.432603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.433425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.434354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.435083 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.435359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.435634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.436014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.436291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.436570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.442989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.443965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.444579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.445551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.447836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.448139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.448531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.448811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.448936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.449093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.449475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.449756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.458165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.458465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.458781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.459026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.459301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.459586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.466953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80130) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.467231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.467505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.467779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80131) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 08:21:14.467897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.468095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.468220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.468372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.468501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.468634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.468762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.468886 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.469026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.469152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.469275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.469409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.469587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.469715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.469841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.469966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.470092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.470232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.470384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.470512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80130) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.470635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.470794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.470995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.471228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80131) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 08:21:14.508845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d45bb2f9c0 pid: 80130 (default app) Jun 29 08:21:14.509614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d45bb2f9c0 pid: 80131 (default app) Jun 29 08:21:14.512344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0de22d-aceb-4f3a-a092-6f59e74bb052 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:14.512637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0de22d-aceb-4f3a-a092-6f59e74bb052 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:14.512918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0de22d-aceb-4f3a-a092-6f59e74bb052 None None] PATH_INFO: `/v3` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:14.517195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Wed Jun 29 08:21:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:22.156940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1e9e-7582-467d-87f6-ba51d5ac138f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:22.157545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1e9e-7582-467d-87f6-ba51d5ac138f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:22.157545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1e9e-7582-467d-87f6-ba51d5ac138f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:22.161957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:21:22.168314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12a9d-c917-4853-88e2-bd52529a3437 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:22.168751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12a9d-c917-4853-88e2-bd52529a3437 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:22.168844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d12a9d-c917-4853-88e2-bd52529a3437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:22.246050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: INFO dbcounter [None req-33d12a9d-c917-4853-88e2-bd52529a3437 None None] Registered counter for database keystone Jun 29 08:21:22.271469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG oslo_db.sqlalchemy.engines [None req-33d12a9d-c917-4853-88e2-bd52529a3437 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80130) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 29 08:21:22.279599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writer thread running {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 29 08:21:22.458349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG passlib.handlers.bcrypt [None req-33d12a9d-c917-4853-88e2-bd52529a3437 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80130) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 29 08:21:22.458752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG passlib.handlers.bcrypt [None req-33d12a9d-c917-4853-88e2-bd52529a3437 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80130) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 29 08:21:22.514369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-33d12a9d-c917-4853-88e2-bd52529a3437 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:22.516025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-33d12a9d-c917-4853-88e2-bd52529a3437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:22.644664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 29 08:21:22.898065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b1e9e-7582-467d-87f6-ba51d5ac138f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:22.978467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: INFO dbcounter [None req-677b1e9e-7582-467d-87f6-ba51d5ac138f None None] Registered counter for database keystone Jun 29 08:21:23.000675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG oslo_db.sqlalchemy.engines [None req-677b1e9e-7582-467d-87f6-ba51d5ac138f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80131) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 29 08:21:23.009769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writer thread running {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 29 08:21:23.128312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b1e9e-7582-467d-87f6-ba51d5ac138f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:23.133907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ef47c-ae19-42e3-b87d-24d5426a67df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:23.134574 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ef47c-ae19-42e3-b87d-24d5426a67df admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:23.134892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ef47c-ae19-42e3-b87d-24d5426a67df admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:23.135166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ef47c-ae19-42e3-b87d-24d5426a67df admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:23.144430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ef47c-ae19-42e3-b87d-24d5426a67df admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:23.196580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ea7ef47c-ae19-42e3-b87d-24d5426a67df admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 29 08:21:23.197157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ef47c-ae19-42e3-b87d-24d5426a67df admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:23.207700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Wed Jun 29 08:21:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:21:23.215189 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d12a9d-c917-4853-88e2-bd52529a3437 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:23.233556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d12a9d-c917-4853-88e2-bd52529a3437 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:23.240177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fafbe-d142-48f0-ac14-ccf09aad1384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:23.241314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fafbe-d142-48f0-ac14-ccf09aad1384 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:23.241605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fafbe-d142-48f0-ac14-ccf09aad1384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:23.241882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fafbe-d142-48f0-ac14-ccf09aad1384 admin admin] PATH_INFO: `/v3/projects/3b8a36f8c21e4c4f9394d7b2d795e615` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:23.243580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fafbe-d142-48f0-ac14-ccf09aad1384 admin admin] RBAC: Authorizing `identity:get_project(project_id=3b8a36f8c21e4c4f9394d7b2d795e615)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:23.295429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e04fafbe-d142-48f0-ac14-ccf09aad1384 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 29 08:21:23.295773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fafbe-d142-48f0-ac14-ccf09aad1384 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:23.300377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Wed Jun 29 08:21:23 2022] GET /identity/v3/projects/3b8a36f8c21e4c4f9394d7b2d795e615 => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:25.318138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83e0ad-ced0-499c-a567-b89e87fd4fed None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:25.318745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83e0ad-ced0-499c-a567-b89e87fd4fed None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:25.318745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83e0ad-ced0-499c-a567-b89e87fd4fed None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:25.322849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:25.329455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d1b7f-69db-49ac-bd68-c21e8ee14980 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:25.330239 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d1b7f-69db-49ac-bd68-c21e8ee14980 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:25.330393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d1b7f-69db-49ac-bd68-c21e8ee14980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:25.358151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-444d1b7f-69db-49ac-bd68-c21e8ee14980 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:25.359735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-444d1b7f-69db-49ac-bd68-c21e8ee14980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:25.369703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 29 08:21:25.392415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83e0ad-ced0-499c-a567-b89e87fd4fed None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:25.407127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83e0ad-ced0-499c-a567-b89e87fd4fed None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:25.412127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b766983-f94d-4da9-b5f0-cc9b29da60fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:25.412765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b766983-f94d-4da9-b5f0-cc9b29da60fe admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:25.413049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b766983-f94d-4da9-b5f0-cc9b29da60fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:25.413322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b766983-f94d-4da9-b5f0-cc9b29da60fe admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:25.421653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b766983-f94d-4da9-b5f0-cc9b29da60fe admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:25.423475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b766983-f94d-4da9-b5f0-cc9b29da60fe admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:25.432799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Wed Jun 29 08:21:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:21:25.438455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d1b7f-69db-49ac-bd68-c21e8ee14980 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:25.448893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d1b7f-69db-49ac-bd68-c21e8ee14980 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:25.454531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2335fa-3be0-445d-ae42-7c3989ab4914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:25.455465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2335fa-3be0-445d-ae42-7c3989ab4914 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:25.455755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2335fa-3be0-445d-ae42-7c3989ab4914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:25.456037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2335fa-3be0-445d-ae42-7c3989ab4914 admin admin] PATH_INFO: `/v3/users/c145cd66ea9b4d3a85c89c24f585acff` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:25.458517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2335fa-3be0-445d-ae42-7c3989ab4914 admin admin] RBAC: Authorizing `identity:get_user(user_id=c145cd66ea9b4d3a85c89c24f585acff)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:25.460079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2335fa-3be0-445d-ae42-7c3989ab4914 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:25.461748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Wed Jun 29 08:21:25 2022] GET /identity/v3/users/c145cd66ea9b4d3a85c89c24f585acff => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:27.531764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cc6b6-3a43-4c8a-a66e-4935ec6849de None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:27.532292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cc6b6-3a43-4c8a-a66e-4935ec6849de None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:27.532292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7cc6b6-3a43-4c8a-a66e-4935ec6849de None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:27.533644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:27.538915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750aa6a-23c7-408f-bd6c-c2a70e228bf8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:27.539353 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750aa6a-23c7-408f-bd6c-c2a70e228bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:27.539447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750aa6a-23c7-408f-bd6c-c2a70e228bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:27.557753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6750aa6a-23c7-408f-bd6c-c2a70e228bf8 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:27.558979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6750aa6a-23c7-408f-bd6c-c2a70e228bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:27.579144 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258b795-1f0c-4dba-95eb-4c6c9a6e1e99 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:27.579430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258b795-1f0c-4dba-95eb-4c6c9a6e1e99 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:27.579734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258b795-1f0c-4dba-95eb-4c6c9a6e1e99 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:27.581109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:27.586759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218409cd-2f53-485c-9d07-16e0919f78c8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:27.587046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218409cd-2f53-485c-9d07-16e0919f78c8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:27.587322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218409cd-2f53-485c-9d07-16e0919f78c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:27.628207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG passlib.handlers.bcrypt [None req-218409cd-2f53-485c-9d07-16e0919f78c8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80131) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 29 08:21:27.628649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG passlib.handlers.bcrypt [None req-218409cd-2f53-485c-9d07-16e0919f78c8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80131) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 29 08:21:27.646117 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:27.669474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750aa6a-23c7-408f-bd6c-c2a70e228bf8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:27.673650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-218409cd-2f53-485c-9d07-16e0919f78c8 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:27.675264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-218409cd-2f53-485c-9d07-16e0919f78c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:27.694027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:27.776650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:21:27.789655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750aa6a-23c7-408f-bd6c-c2a70e228bf8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:27.798842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218409cd-2f53-485c-9d07-16e0919f78c8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:27.825924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859da556-b9a8-4ad8-8dc2-cb27f1f3b841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:27.826897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859da556-b9a8-4ad8-8dc2-cb27f1f3b841 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:27.827179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859da556-b9a8-4ad8-8dc2-cb27f1f3b841 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:27.827453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859da556-b9a8-4ad8-8dc2-cb27f1f3b841 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:27.839789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859da556-b9a8-4ad8-8dc2-cb27f1f3b841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-859da556-b9a8-4ad8-8dc2-cb27f1f3b841 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:27.851085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:27.852044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:27.852862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:21:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:21:27.858164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859da556-b9a8-4ad8-8dc2-cb27f1f3b841 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:27.901023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218409cd-2f53-485c-9d07-16e0919f78c8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:27.903416 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:27.939864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816da29-cf62-4e4f-84de-48bc92f7b171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:27.940781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816da29-cf62-4e4f-84de-48bc92f7b171 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:27.941073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816da29-cf62-4e4f-84de-48bc92f7b171 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:27.941600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816da29-cf62-4e4f-84de-48bc92f7b171 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:27.943480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:27.963754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859da556-b9a8-4ad8-8dc2-cb27f1f3b841 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:27.997629 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e1308-1e33-4ada-86dc-4fa0f5bfda01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:27.998308 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e1308-1e33-4ada-86dc-4fa0f5bfda01 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:27.998593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e1308-1e33-4ada-86dc-4fa0f5bfda01 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:27.998910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e1308-1e33-4ada-86dc-4fa0f5bfda01 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:27.999686 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e1308-1e33-4ada-86dc-4fa0f5bfda01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:27.999996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e1308-1e33-4ada-86dc-4fa0f5bfda01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:28.001448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e1308-1e33-4ada-86dc-4fa0f5bfda01 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:28.009117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816da29-cf62-4e4f-84de-48bc92f7b171 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:28.010653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816da29-cf62-4e4f-84de-48bc92f7b171 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:28.013568 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:28.018983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e1308-1e33-4ada-86dc-4fa0f5bfda01 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:28.109327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 08:21:28.115653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816da29-cf62-4e4f-84de-48bc92f7b171 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:28.118654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:28.121760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e1308-1e33-4ada-86dc-4fa0f5bfda01 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:28.154984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059e17-632c-46c1-87e8-3e41a389428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:28.155891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059e17-632c-46c1-87e8-3e41a389428d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:28.156194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059e17-632c-46c1-87e8-3e41a389428d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:28.156457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059e17-632c-46c1-87e8-3e41a389428d None admin] PATH_INFO: `/v3/users/c145cd66ea9b4d3a85c89c24f585acff` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:28.157685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74059e17-632c-46c1-87e8-3e41a389428d None admin] RBAC: Authorizing `identity:get_user(user_id=c145cd66ea9b4d3a85c89c24f585acff)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:28.159195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74059e17-632c-46c1-87e8-3e41a389428d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:28.160890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:28 2022] GET /identity/v3/users/c145cd66ea9b4d3a85c89c24f585acff => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:28.166330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059e17-632c-46c1-87e8-3e41a389428d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:28.180991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:28.216176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816da29-cf62-4e4f-84de-48bc92f7b171 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:28.251488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f238ff-d60d-48b8-b40a-7da93eb0a820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:28.252299 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f238ff-d60d-48b8-b40a-7da93eb0a820 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:28.252563 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f238ff-d60d-48b8-b40a-7da93eb0a820 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:28.252853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f238ff-d60d-48b8-b40a-7da93eb0a820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:28.266961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059e17-632c-46c1-87e8-3e41a389428d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:28.271836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f238ff-d60d-48b8-b40a-7da93eb0a820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:28.273331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f238ff-d60d-48b8-b40a-7da93eb0a820 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-54f238ff-d60d-48b8-b40a-7da93eb0a820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:21:28.293620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:28.294429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:28.295272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:21:28.295272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:21:28.295272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:21:28.295272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:21:28.295272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 9/18] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:28.300707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f238ff-d60d-48b8-b40a-7da93eb0a820 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:28.303808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570502d-5b13-4667-a6fc-a7d3cbc4db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:28.304893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570502d-5b13-4667-a6fc-a7d3cbc4db17 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:28.305174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570502d-5b13-4667-a6fc-a7d3cbc4db17 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:28.305455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5570502d-5b13-4667-a6fc-a7d3cbc4db17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:28.306714 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570502d-5b13-4667-a6fc-a7d3cbc4db17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:28.308167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5570502d-5b13-4667-a6fc-a7d3cbc4db17 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:28.309788 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 10/19] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:28.315562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570502d-5b13-4667-a6fc-a7d3cbc4db17 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:28.359313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:28.399259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:28.404922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f238ff-d60d-48b8-b40a-7da93eb0a820 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:28.419429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5570502d-5b13-4667-a6fc-a7d3cbc4db17 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:28.439709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf7a5c-163e-4c25-9668-c905c440bc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:28.440487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7a5c-163e-4c25-9668-c905c440bc27 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:28.440841 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7a5c-163e-4c25-9668-c905c440bc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:28.441195 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf7a5c-163e-4c25-9668-c905c440bc27 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:28.442105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf7a5c-163e-4c25-9668-c905c440bc27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:28.442490 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf7a5c-163e-4c25-9668-c905c440bc27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:28.443948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf7a5c-163e-4c25-9668-c905c440bc27 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:28.454099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972070d-e915-4d79-8dd1-25aa8f4d980a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:28.454992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972070d-e915-4d79-8dd1-25aa8f4d980a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:28.455259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972070d-e915-4d79-8dd1-25aa8f4d980a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:28.455536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972070d-e915-4d79-8dd1-25aa8f4d980a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:28.456382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972070d-e915-4d79-8dd1-25aa8f4d980a None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.domain.id=default, user.id=c145cd66ea9b4d3a85c89c24f585acff) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:28.456695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972070d-e915-4d79-8dd1-25aa8f4d980a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:28.458353 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972070d-e915-4d79-8dd1-25aa8f4d980a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:28.462440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 10/20] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:28.466360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 11/21] 10.222.0.27 () {56 vars in 1322 bytes} [Wed Jun 29 08:21:28 2022] GET /identity/v3/role_assignments?user.id=c145cd66ea9b4d3a85c89c24f585acff&scope.domain.id=default&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:30.739597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f3ec9-7fcd-49aa-89f7-c7a1acf67afb None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:30.740608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f3ec9-7fcd-49aa-89f7-c7a1acf67afb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:30.741172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f3ec9-7fcd-49aa-89f7-c7a1acf67afb None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:30.742774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 11/22] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:30.748286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b9f1b-7e0f-4a6f-b62b-17345f15e1fa None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:30.749082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b9f1b-7e0f-4a6f-b62b-17345f15e1fa None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:30.749549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b9f1b-7e0f-4a6f-b62b-17345f15e1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:30.775854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-160b9f1b-7e0f-4a6f-b62b-17345f15e1fa None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:30.777430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-160b9f1b-7e0f-4a6f-b62b-17345f15e1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:30.864489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 12/23] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:30.898903 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f3ec9-7fcd-49aa-89f7-c7a1acf67afb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:31.033362 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083b6d-fefd-46d8-b500-2933af34b14a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.033550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083b6d-fefd-46d8-b500-2933af34b14a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.033833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083b6d-fefd-46d8-b500-2933af34b14a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.035515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 13/24] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:31.046993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c017c1-73d6-4f1b-ac3e-d3ec8629e9ab None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.047280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c017c1-73d6-4f1b-ac3e-d3ec8629e9ab None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.047562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c017c1-73d6-4f1b-ac3e-d3ec8629e9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.053774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f3ec9-7fcd-49aa-89f7-c7a1acf67afb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:31.074947 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-21c017c1-73d6-4f1b-ac3e-d3ec8629e9ab None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:31.076356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-21c017c1-73d6-4f1b-ac3e-d3ec8629e9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:31.125698 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:31.176915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 14/25] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:21:31.182011 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249c345-570e-4de3-b496-d94b4949c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:31.183016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249c345-570e-4de3-b496-d94b4949c603 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.183428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249c345-570e-4de3-b496-d94b4949c603 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.183833 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249c345-570e-4de3-b496-d94b4949c603 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.195702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249c345-570e-4de3-b496-d94b4949c603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c249c345-570e-4de3-b496-d94b4949c603 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:31.204661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:31.205666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:31.206579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:21:31.208261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c017c1-73d6-4f1b-ac3e-d3ec8629e9ab None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:31.219020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:31.222104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 12/26] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:21:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:31.227088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249c345-570e-4de3-b496-d94b4949c603 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:31.373017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249c345-570e-4de3-b496-d94b4949c603 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:31.412667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf6119d-aa28-4b4d-9cbc-47ae57fd2532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:31.413579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6119d-aa28-4b4d-9cbc-47ae57fd2532 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.414023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6119d-aa28-4b4d-9cbc-47ae57fd2532 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.414441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf6119d-aa28-4b4d-9cbc-47ae57fd2532 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.415613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6119d-aa28-4b4d-9cbc-47ae57fd2532 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:31.416055 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6119d-aa28-4b4d-9cbc-47ae57fd2532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:31.418105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf6119d-aa28-4b4d-9cbc-47ae57fd2532 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:31.430544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 13/27] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:31.432325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f22c8-5ed1-4074-ba77-f9f9587f7e90 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.433145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f22c8-5ed1-4074-ba77-f9f9587f7e90 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.433590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f22c8-5ed1-4074-ba77-f9f9587f7e90 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.435304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 14/28] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:31.436151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f22c8-5ed1-4074-ba77-f9f9587f7e90 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:31.448004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c017c1-73d6-4f1b-ac3e-d3ec8629e9ab None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:31.483056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:31.548251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f22c8-5ed1-4074-ba77-f9f9587f7e90 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:31.571440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a08cfa-1204-4cf7-bdbd-dde3d864fea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:31.572279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a08cfa-1204-4cf7-bdbd-dde3d864fea7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.572726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a08cfa-1204-4cf7-bdbd-dde3d864fea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.573155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a08cfa-1204-4cf7-bdbd-dde3d864fea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.584197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4f369-bbfa-4fbe-9fbb-c0c61d4bfe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:31.585468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f369-bbfa-4fbe-9fbb-c0c61d4bfe8e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.585982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f369-bbfa-4fbe-9fbb-c0c61d4bfe8e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.586446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4f369-bbfa-4fbe-9fbb-c0c61d4bfe8e None admin] PATH_INFO: `/v3/users/c145cd66ea9b4d3a85c89c24f585acff` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.588564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4f369-bbfa-4fbe-9fbb-c0c61d4bfe8e None admin] RBAC: Authorizing `identity:get_user(user_id=c145cd66ea9b4d3a85c89c24f585acff)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:31.590329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4f369-bbfa-4fbe-9fbb-c0c61d4bfe8e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:31.592292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 15/29] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:31 2022] GET /identity/v3/users/c145cd66ea9b4d3a85c89c24f585acff => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:31.593632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25f3b0-40be-49cb-9357-7aeed0aeefd2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.594114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25f3b0-40be-49cb-9357-7aeed0aeefd2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.594573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25f3b0-40be-49cb-9357-7aeed0aeefd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.597749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a08cfa-1204-4cf7-bdbd-dde3d864fea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:31.599303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a08cfa-1204-4cf7-bdbd-dde3d864fea7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:31.612233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3d25f3b0-40be-49cb-9357-7aeed0aeefd2 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:31.613874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3d25f3b0-40be-49cb-9357-7aeed0aeefd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-11a08cfa-1204-4cf7-bdbd-dde3d864fea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:21:31.634912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:31.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:31.636509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:21:31.636509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:21:31.636509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:21:31.636509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:31.638507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 15/30] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:31.639809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a08cfa-1204-4cf7-bdbd-dde3d864fea7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:31.683683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 16/31] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:31.685116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6ba4f-12c9-431a-927d-b687f210d73d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.685407 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6ba4f-12c9-431a-927d-b687f210d73d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.685677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d6ba4f-12c9-431a-927d-b687f210d73d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.687051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 17/32] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:31.687870 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6ba4f-12c9-431a-927d-b687f210d73d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:31.700525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:31.743209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:31.759858 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a08cfa-1204-4cf7-bdbd-dde3d864fea7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:31.798226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05734f3-91c4-4028-8f03-d028fa8d58b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:31.799338 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05734f3-91c4-4028-8f03-d028fa8d58b4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.799790 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05734f3-91c4-4028-8f03-d028fa8d58b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.800204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05734f3-91c4-4028-8f03-d028fa8d58b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.801644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05734f3-91c4-4028-8f03-d028fa8d58b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:31.803342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05734f3-91c4-4028-8f03-d028fa8d58b4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:31.805283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 16/33] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:31.806825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207209ee-3c33-4996-9feb-5bcf400117ed None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.807329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207209ee-3c33-4996-9feb-5bcf400117ed None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.807779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207209ee-3c33-4996-9feb-5bcf400117ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.825868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-207209ee-3c33-4996-9feb-5bcf400117ed None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:31.827473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-207209ee-3c33-4996-9feb-5bcf400117ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:31.829770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d6ba4f-12c9-431a-927d-b687f210d73d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:31.873236 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779a775-dd11-4b6f-aa2f-db11e6f3cfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:31.874258 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779a775-dd11-4b6f-aa2f-db11e6f3cfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:31.875408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779a775-dd11-4b6f-aa2f-db11e6f3cfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:31.875876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779a775-dd11-4b6f-aa2f-db11e6f3cfa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:31.877210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779a775-dd11-4b6f-aa2f-db11e6f3cfa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:31.877912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779a775-dd11-4b6f-aa2f-db11e6f3cfa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:31.879915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779a775-dd11-4b6f-aa2f-db11e6f3cfa5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:31.896708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 18/34] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:31.898205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779a775-dd11-4b6f-aa2f-db11e6f3cfa5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:31.902823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 17/35] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:31.903616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207209ee-3c33-4996-9feb-5bcf400117ed None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:31.974508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:32.015141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:32.021540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779a775-dd11-4b6f-aa2f-db11e6f3cfa5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:32.033100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207209ee-3c33-4996-9feb-5bcf400117ed None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:32.058064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fcba2-3780-469e-a48d-c6a44a05a338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:32.058781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fcba2-3780-469e-a48d-c6a44a05a338 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:32.059067 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fcba2-3780-469e-a48d-c6a44a05a338 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:32.059659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fcba2-3780-469e-a48d-c6a44a05a338 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:32.060098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fcba2-3780-469e-a48d-c6a44a05a338 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:32.061506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02fcba2-3780-469e-a48d-c6a44a05a338 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:32.075341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397efa0-f956-4e91-a241-c9581ac4137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:32.076523 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397efa0-f956-4e91-a241-c9581ac4137f None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:32.076799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397efa0-f956-4e91-a241-c9581ac4137f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:32.077071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397efa0-f956-4e91-a241-c9581ac4137f None admin] PATH_INFO: `/v3/domains/default/users/c145cd66ea9b4d3a85c89c24f585acff/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:32.079138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397efa0-f956-4e91-a241-c9581ac4137f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c145cd66ea9b4d3a85c89c24f585acff, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:32.080578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397efa0-f956-4e91-a241-c9581ac4137f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:32.185665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 19/36] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Jun 29 08:21:31 2022] POST /identity/v3/roles => generated 219 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:21:32.186385 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fcba2-3780-469e-a48d-c6a44a05a338 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:32.203042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 18/37] 10.222.0.27 () {58 vars in 1361 bytes} [Wed Jun 29 08:21:31 2022] PUT /identity/v3/domains/default/users/c145cd66ea9b4d3a85c89c24f585acff/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:21:32.205015 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397efa0-f956-4e91-a241-c9581ac4137f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:32.311375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fcba2-3780-469e-a48d-c6a44a05a338 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:32.363054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c9afb-5a49-4c08-8837-6f5e63f82b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:32.363853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c9afb-5a49-4c08-8837-6f5e63f82b73 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:32.364347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c9afb-5a49-4c08-8837-6f5e63f82b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:32.364807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c9afb-5a49-4c08-8837-6f5e63f82b73 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:32.366236 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c9afb-5a49-4c08-8837-6f5e63f82b73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:32.368373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c9afb-5a49-4c08-8837-6f5e63f82b73 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:32.389982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397efa0-f956-4e91-a241-c9581ac4137f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:32.404808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 20/38] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:21:32 2022] POST /identity/v3/projects => generated 301 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:21:32.406451 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c5b78-1442-40a1-81bb-73ef736058b8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:32.406789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c5b78-1442-40a1-81bb-73ef736058b8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:32.407124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c5b78-1442-40a1-81bb-73ef736058b8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:32.408653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 21/39] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:32.410077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a14258-f4b0-483d-b390-cac8e7c44abe None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:32.410331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a14258-f4b0-483d-b390-cac8e7c44abe None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:32.410610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a14258-f4b0-483d-b390-cac8e7c44abe None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:32.415971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:32.416411 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 22/40] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:32.418633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d49f13-94be-4d57-b951-22809584abdc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:32.419160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d49f13-94be-4d57-b951-22809584abdc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:32.419444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d49f13-94be-4d57-b951-22809584abdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:32.443659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e5d49f13-94be-4d57-b951-22809584abdc None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:32.446445 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e5d49f13-94be-4d57-b951-22809584abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:32.449841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb5852-eaa9-4b28-8089-8ffb931c5005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:32.450019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:32.450442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb5852-eaa9-4b28-8089-8ffb931c5005 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:32.450845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb5852-eaa9-4b28-8089-8ffb931c5005 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:32.451163 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb5852-eaa9-4b28-8089-8ffb931c5005 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:32.454125 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb5852-eaa9-4b28-8089-8ffb931c5005 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:32.455913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb5852-eaa9-4b28-8089-8ffb931c5005 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:32.486120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 19/41] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Jun 29 08:21:32 2022] POST /identity/v3/roles => generated 225 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:21:32.487513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c94578-1548-4592-934d-2e2a65479f19 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:32.487898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c94578-1548-4592-934d-2e2a65479f19 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:32.488361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c94578-1548-4592-934d-2e2a65479f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:32.564324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 23/42] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:21:32.606886 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d49f13-94be-4d57-b951-22809584abdc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:32.631752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-21c94578-1548-4592-934d-2e2a65479f19 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:32.633414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-21c94578-1548-4592-934d-2e2a65479f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:32.712583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 20/43] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:32.736537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c94578-1548-4592-934d-2e2a65479f19 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:32.790008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d49f13-94be-4d57-b951-22809584abdc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:32.840313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf6239-493f-430c-acfb-2717d4d81081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:32.841102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf6239-493f-430c-acfb-2717d4d81081 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:32.841386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf6239-493f-430c-acfb-2717d4d81081 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:32.841663 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf6239-493f-430c-acfb-2717d4d81081 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:32.842437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf6239-493f-430c-acfb-2717d4d81081 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:32.843868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf6239-493f-430c-acfb-2717d4d81081 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:32.853125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:32.859450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 24/44] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Jun 29 08:21:32 2022] POST /identity/v3/roles => generated 223 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:21:32.876084 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c94578-1548-4592-934d-2e2a65479f19 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:32.880494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:32.919195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dda1da-c74c-4edf-8727-25922bc27443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:32.920012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dda1da-c74c-4edf-8727-25922bc27443 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:32.920323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dda1da-c74c-4edf-8727-25922bc27443 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:32.920579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dda1da-c74c-4edf-8727-25922bc27443 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:32.921917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dda1da-c74c-4edf-8727-25922bc27443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:32.923460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dda1da-c74c-4edf-8727-25922bc27443 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:32.925378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 21/45] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:32.931956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf6239-493f-430c-acfb-2717d4d81081 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:33.153693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf6239-493f-430c-acfb-2717d4d81081 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:33.188320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97770e-040a-421f-8924-34b443fbe46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:33.188985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97770e-040a-421f-8924-34b443fbe46e None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:33.189272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97770e-040a-421f-8924-34b443fbe46e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:33.189548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97770e-040a-421f-8924-34b443fbe46e None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:33.190266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97770e-040a-421f-8924-34b443fbe46e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:33.192122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97770e-040a-421f-8924-34b443fbe46e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:33.224939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 25/46] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:21:32 2022] POST /identity/v3/projects => generated 312 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:21:34.431847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc9209-dfb2-42d6-b2ff-5e5acf648ee7 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:34.432375 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc9209-dfb2-42d6-b2ff-5e5acf648ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:34.432459 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc9209-dfb2-42d6-b2ff-5e5acf648ee7 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:34.433849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 22/47] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:34.439191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378658d-ffa0-4ea0-9899-5e7f2994653d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:34.439612 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378658d-ffa0-4ea0-9899-5e7f2994653d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:34.439788 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378658d-ffa0-4ea0-9899-5e7f2994653d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:34.459442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b378658d-ffa0-4ea0-9899-5e7f2994653d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:34.460684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b378658d-ffa0-4ea0-9899-5e7f2994653d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:34.513597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:34.537757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 26/48] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:34.561564 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc9209-dfb2-42d6-b2ff-5e5acf648ee7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:34.668483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc9209-dfb2-42d6-b2ff-5e5acf648ee7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:34.704482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd9315-af0a-428b-ae6f-4db3fb083479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:34.705208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd9315-af0a-428b-ae6f-4db3fb083479 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:34.705492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd9315-af0a-428b-ae6f-4db3fb083479 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:34.705764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd9315-af0a-428b-ae6f-4db3fb083479 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:34.716752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd9315-af0a-428b-ae6f-4db3fb083479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e4dd9315-af0a-428b-ae6f-4db3fb083479 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:34.724886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:34.725735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:34.726513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 23/49] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:21:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:34.732254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378658d-ffa0-4ea0-9899-5e7f2994653d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:34.833104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b378658d-ffa0-4ea0-9899-5e7f2994653d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:34.868961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c93850-0fc6-413e-a302-d92e7c14b850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:34.869633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c93850-0fc6-413e-a302-d92e7c14b850 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:34.869919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c93850-0fc6-413e-a302-d92e7c14b850 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:34.870200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c93850-0fc6-413e-a302-d92e7c14b850 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:34.870976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c93850-0fc6-413e-a302-d92e7c14b850 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:34.871288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c93850-0fc6-413e-a302-d92e7c14b850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:34.872696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c93850-0fc6-413e-a302-d92e7c14b850 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:34.881201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 27/50] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:34.886967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd9315-af0a-428b-ae6f-4db3fb083479 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:34.919852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:34.993051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd9315-af0a-428b-ae6f-4db3fb083479 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:35.028229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac112e-1864-41f5-b700-714de0102365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:35.029116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac112e-1864-41f5-b700-714de0102365 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:35.029406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac112e-1864-41f5-b700-714de0102365 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:35.029685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac112e-1864-41f5-b700-714de0102365 None admin] PATH_INFO: `/v3/users/c145cd66ea9b4d3a85c89c24f585acff` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:35.030965 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac112e-1864-41f5-b700-714de0102365 None admin] RBAC: Authorizing `identity:get_user(user_id=c145cd66ea9b4d3a85c89c24f585acff)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:35.032432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac112e-1864-41f5-b700-714de0102365 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:35.034290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 24/51] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:34 2022] GET /identity/v3/users/c145cd66ea9b4d3a85c89c24f585acff => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:35.039954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c93850-0fc6-413e-a302-d92e7c14b850 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:35.085554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:35.148076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c93850-0fc6-413e-a302-d92e7c14b850 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:35.185368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda9343-7a50-42e7-a26d-3f08fd36f631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:35.186181 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda9343-7a50-42e7-a26d-3f08fd36f631 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:35.186469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda9343-7a50-42e7-a26d-3f08fd36f631 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:35.186745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda9343-7a50-42e7-a26d-3f08fd36f631 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:35.188057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda9343-7a50-42e7-a26d-3f08fd36f631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:35.189552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eda9343-7a50-42e7-a26d-3f08fd36f631 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:35.191713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 28/52] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:35.197429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac112e-1864-41f5-b700-714de0102365 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:35.217844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0565b7c-6aa3-4a81-90e7-5164c2aa36ae None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:35.218302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0565b7c-6aa3-4a81-90e7-5164c2aa36ae None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:35.218816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0565b7c-6aa3-4a81-90e7-5164c2aa36ae None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:35.221332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 29/53] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:21:35.227231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88f150-554f-4127-a77d-e7a115ab5b12 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:35.227518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88f150-554f-4127-a77d-e7a115ab5b12 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:35.227799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88f150-554f-4127-a77d-e7a115ab5b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:35.245593 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0f88f150-554f-4127-a77d-e7a115ab5b12 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:35.246762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0f88f150-554f-4127-a77d-e7a115ab5b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:35.298704 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:35.302352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac112e-1864-41f5-b700-714de0102365 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:35.314538 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 30/54] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:35.336815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c5aaf-de5e-402b-9aaa-559d797cd63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:35.337478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c5aaf-de5e-402b-9aaa-559d797cd63b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:35.337761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c5aaf-de5e-402b-9aaa-559d797cd63b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:35.337889 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88f150-554f-4127-a77d-e7a115ab5b12 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:35.338015 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c5aaf-de5e-402b-9aaa-559d797cd63b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:35.339264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c5aaf-de5e-402b-9aaa-559d797cd63b None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.domain.id=default, user.id=c145cd66ea9b4d3a85c89c24f585acff) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:35.339573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c5aaf-de5e-402b-9aaa-559d797cd63b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:35.341023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c5aaf-de5e-402b-9aaa-559d797cd63b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:35.347156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 25/55] 10.222.0.27 () {56 vars in 1322 bytes} [Wed Jun 29 08:21:35 2022] GET /identity/v3/role_assignments?user.id=c145cd66ea9b4d3a85c89c24f585acff&scope.domain.id=default&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:35.383670 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:35.445887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88f150-554f-4127-a77d-e7a115ab5b12 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:35.484311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315e61b-9d51-48d9-9e11-0ba71123562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:35.485136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315e61b-9d51-48d9-9e11-0ba71123562a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:35.485421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315e61b-9d51-48d9-9e11-0ba71123562a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:35.485698 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315e61b-9d51-48d9-9e11-0ba71123562a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:35.486979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315e61b-9d51-48d9-9e11-0ba71123562a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:35.488691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315e61b-9d51-48d9-9e11-0ba71123562a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:35.490669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 31/56] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:35.496518 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c5aaf-de5e-402b-9aaa-559d797cd63b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:35.599586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c5aaf-de5e-402b-9aaa-559d797cd63b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:35.632030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed10bfe-fe39-49ee-abc6-8de4127a3fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:35.632690 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10bfe-fe39-49ee-abc6-8de4127a3fc2 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:35.632969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10bfe-fe39-49ee-abc6-8de4127a3fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:35.633246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10bfe-fe39-49ee-abc6-8de4127a3fc2 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:35.633966 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed10bfe-fe39-49ee-abc6-8de4127a3fc2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:35.635440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed10bfe-fe39-49ee-abc6-8de4127a3fc2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:35.666218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 26/57] 10.222.0.27 () {60 vars in 1140 bytes} [Wed Jun 29 08:21:35 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:21:37.683546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610fad6-48cc-4328-b618-ac658be5ec32 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:37.684085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610fad6-48cc-4328-b618-ac658be5ec32 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:37.684085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e610fad6-48cc-4328-b618-ac658be5ec32 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:37.685499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 32/58] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:37.691982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7ab08-caef-4d44-bdd5-3990a199fe1c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:37.692658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7ab08-caef-4d44-bdd5-3990a199fe1c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:37.692872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7ab08-caef-4d44-bdd5-3990a199fe1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:37.717248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0ec7ab08-caef-4d44-bdd5-3990a199fe1c None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:37.718381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0ec7ab08-caef-4d44-bdd5-3990a199fe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:37.730426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:37.791827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 27/59] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:37.819205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610fad6-48cc-4328-b618-ac658be5ec32 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:37.930463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e610fad6-48cc-4328-b618-ac658be5ec32 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:37.939679 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:37.968771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9f5b3-eb75-4f88-bfdb-604ea0ea0814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:37.969499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f5b3-eb75-4f88-bfdb-604ea0ea0814 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:37.969783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f5b3-eb75-4f88-bfdb-604ea0ea0814 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:37.970057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f5b3-eb75-4f88-bfdb-604ea0ea0814 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:37.971333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9f5b3-eb75-4f88-bfdb-604ea0ea0814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:37.972867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9f5b3-eb75-4f88-bfdb-604ea0ea0814 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:37.974477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 33/60] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:37.980235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7ab08-caef-4d44-bdd5-3990a199fe1c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:38.090413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7ab08-caef-4d44-bdd5-3990a199fe1c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:38.129305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56086ec4-d54e-44ac-8619-74a92b1bdfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:38.129971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56086ec4-d54e-44ac-8619-74a92b1bdfaf None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:38.130260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56086ec4-d54e-44ac-8619-74a92b1bdfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:38.130537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56086ec4-d54e-44ac-8619-74a92b1bdfaf None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:38.131287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56086ec4-d54e-44ac-8619-74a92b1bdfaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:38.132741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56086ec4-d54e-44ac-8619-74a92b1bdfaf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:38.163335 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 28/61] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:21:37 2022] POST /identity/v3/users => generated 275 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:21:41.352688 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5126e56-5591-4210-adb6-13df9408b003 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:41.353829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5126e56-5591-4210-adb6-13df9408b003 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:41.354305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5126e56-5591-4210-adb6-13df9408b003 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:41.355918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 34/62] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:41.361429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430d5e3-f86f-4fdb-a996-e3d0e17669b8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:41.362259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430d5e3-f86f-4fdb-a996-e3d0e17669b8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:41.362808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430d5e3-f86f-4fdb-a996-e3d0e17669b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:41.369117 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:41.388047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4430d5e3-f86f-4fdb-a996-e3d0e17669b8 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:41.389259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4430d5e3-f86f-4fdb-a996-e3d0e17669b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:41.487440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 29/63] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:41.496126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a3d35-5bb6-45ec-a16d-906c25a8d0ee None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:41.496403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a3d35-5bb6-45ec-a16d-906c25a8d0ee None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:41.496922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a3d35-5bb6-45ec-a16d-906c25a8d0ee None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:41.498309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 35/64] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:41.503759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0170b-b3b7-4356-bf0e-c6e0163d4e4a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:41.504239 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0170b-b3b7-4356-bf0e-c6e0163d4e4a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:41.504655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac0170b-b3b7-4356-bf0e-c6e0163d4e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:41.526935 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9ac0170b-b3b7-4356-bf0e-c6e0163d4e4a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:41.528352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9ac0170b-b3b7-4356-bf0e-c6e0163d4e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:41.535472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a3d35-5bb6-45ec-a16d-906c25a8d0ee None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:41.627893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 30/65] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:41.629644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0115a1a-7be7-4d59-8302-032b6503de54 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:41.630075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0115a1a-7be7-4d59-8302-032b6503de54 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:41.630487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0115a1a-7be7-4d59-8302-032b6503de54 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:41.631945 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 31/66] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:41.645001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87964f2e-8748-4f09-8da2-936f3a3e45fb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:41.645471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87964f2e-8748-4f09-8da2-936f3a3e45fb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:41.645883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87964f2e-8748-4f09-8da2-936f3a3e45fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:41.662812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-87964f2e-8748-4f09-8da2-936f3a3e45fb None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:41.664270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-87964f2e-8748-4f09-8da2-936f3a3e45fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:41.679996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a3d35-5bb6-45ec-a16d-906c25a8d0ee None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:41.721463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aafa4-343d-4f67-9d80-f52f85452177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:41.723346 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aafa4-343d-4f67-9d80-f52f85452177 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:41.723809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aafa4-343d-4f67-9d80-f52f85452177 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:41.724499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aafa4-343d-4f67-9d80-f52f85452177 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:41.734367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:41.737402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447aafa4-343d-4f67-9d80-f52f85452177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:41.746634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 32/67] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-447aafa4-343d-4f67-9d80-f52f85452177 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:41.747365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:41.748173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:41.748913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:21:41.749474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87964f2e-8748-4f09-8da2-936f3a3e45fb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:41.752371 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 36/68] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:21:41.758344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aafa4-343d-4f67-9d80-f52f85452177 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:41.800619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:41.870315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87964f2e-8748-4f09-8da2-936f3a3e45fb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:41.872247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aafa4-343d-4f67-9d80-f52f85452177 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:41.903936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376affe-109b-44dc-9849-678bb09d6752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:41.904696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376affe-109b-44dc-9849-678bb09d6752 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:41.904981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376affe-109b-44dc-9849-678bb09d6752 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:41.905258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376affe-109b-44dc-9849-678bb09d6752 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:41.905673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f27ff4-9dc1-4a1f-aeb8-6dae0e227f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:41.906344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f27ff4-9dc1-4a1f-aeb8-6dae0e227f5e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:41.906627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f27ff4-9dc1-4a1f-aeb8-6dae0e227f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:41.906935 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f27ff4-9dc1-4a1f-aeb8-6dae0e227f5e None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:41.907695 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f27ff4-9dc1-4a1f-aeb8-6dae0e227f5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:41.908006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f27ff4-9dc1-4a1f-aeb8-6dae0e227f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:41.909406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f27ff4-9dc1-4a1f-aeb8-6dae0e227f5e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:41.917717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 37/69] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:21:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:41.918339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f27ff4-9dc1-4a1f-aeb8-6dae0e227f5e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:41.919558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376affe-109b-44dc-9849-678bb09d6752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f376affe-109b-44dc-9849-678bb09d6752 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:41.927273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:41.928142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:41.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 33/70] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:21:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:41.930031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374fbb-2b4d-4e21-b45e-8f4c17450847 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:41.930131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374fbb-2b4d-4e21-b45e-8f4c17450847 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:41.930385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374fbb-2b4d-4e21-b45e-8f4c17450847 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:41.931816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 34/71] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:41.934381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b8dc5-79f5-4791-8a92-da7b059fe4fe None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:41.934950 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b8dc5-79f5-4791-8a92-da7b059fe4fe None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:41.935232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b8dc5-79f5-4791-8a92-da7b059fe4fe None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:41.936544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 35/72] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:41.937705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b8dc5-79f5-4791-8a92-da7b059fe4fe None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:41.992399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:42.023605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:42.027323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f27ff4-9dc1-4a1f-aeb8-6dae0e227f5e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:42.044944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b8dc5-79f5-4791-8a92-da7b059fe4fe None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:42.064395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6df7b4-e5ea-4b34-b064-919ffce20554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:42.065197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6df7b4-e5ea-4b34-b064-919ffce20554 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.065479 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6df7b4-e5ea-4b34-b064-919ffce20554 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.065752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6df7b4-e5ea-4b34-b064-919ffce20554 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.075925 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6df7b4-e5ea-4b34-b064-919ffce20554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:42.078759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05575df-89fa-48e9-b541-2f064bacb92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:42.079652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05575df-89fa-48e9-b541-2f064bacb92e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.079941 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05575df-89fa-48e9-b541-2f064bacb92e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.080220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05575df-89fa-48e9-b541-2f064bacb92e None admin] PATH_INFO: `/v3/users/c2a06e0093704ec0865c012a4c2427dc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3a6df7b4-e5ea-4b34-b064-919ffce20554 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:42.084182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:42.085025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:21:42.085768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 38/73] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:21:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:42.086378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6df7b4-e5ea-4b34-b064-919ffce20554 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:42.095092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05575df-89fa-48e9-b541-2f064bacb92e None admin] RBAC: Authorizing `identity:get_user(user_id=c2a06e0093704ec0865c012a4c2427dc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:42.096580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05575df-89fa-48e9-b541-2f064bacb92e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:42.098363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 36/74] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:41 2022] GET /identity/v3/users/c2a06e0093704ec0865c012a4c2427dc => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:42.101109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21246d3-5479-4c6a-9fa8-76249231e3b0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.101420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21246d3-5479-4c6a-9fa8-76249231e3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.102060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21246d3-5479-4c6a-9fa8-76249231e3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.118977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e21246d3-5479-4c6a-9fa8-76249231e3b0 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:42.120221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e21246d3-5479-4c6a-9fa8-76249231e3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:42.186147 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 37/75] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:42.187384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a99ee-ddad-4c90-a790-949526095f2f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.187686 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a99ee-ddad-4c90-a790-949526095f2f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.187963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a99ee-ddad-4c90-a790-949526095f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.190509 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6df7b4-e5ea-4b34-b064-919ffce20554 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:42.215370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-948a99ee-ddad-4c90-a790-949526095f2f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:42.216564 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-948a99ee-ddad-4c90-a790-949526095f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:42.224959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452ce4e-3cdd-4dda-93b4-f70be8b8d08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:42.225631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452ce4e-3cdd-4dda-93b4-f70be8b8d08d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.225916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452ce4e-3cdd-4dda-93b4-f70be8b8d08d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.226191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452ce4e-3cdd-4dda-93b4-f70be8b8d08d None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.226987 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452ce4e-3cdd-4dda-93b4-f70be8b8d08d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:42.227298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452ce4e-3cdd-4dda-93b4-f70be8b8d08d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:42.228702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452ce4e-3cdd-4dda-93b4-f70be8b8d08d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:42.237255 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 39/76] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:42.238100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452ce4e-3cdd-4dda-93b4-f70be8b8d08d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:42.252647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:42.260615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:42.286610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 38/77] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:42.287471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a99ee-ddad-4c90-a790-949526095f2f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:42.343823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452ce4e-3cdd-4dda-93b4-f70be8b8d08d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:42.380376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60946e8e-d4f0-4ae0-b785-ff09cc3525b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:42.381058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60946e8e-d4f0-4ae0-b785-ff09cc3525b6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.381329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60946e8e-d4f0-4ae0-b785-ff09cc3525b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.381606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60946e8e-d4f0-4ae0-b785-ff09cc3525b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.382368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60946e8e-d4f0-4ae0-b785-ff09cc3525b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:42.382683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60946e8e-d4f0-4ae0-b785-ff09cc3525b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:42.384119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60946e8e-d4f0-4ae0-b785-ff09cc3525b6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:42.392747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 40/78] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:21:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:42.395561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60946e8e-d4f0-4ae0-b785-ff09cc3525b6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:42.400024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a99ee-ddad-4c90-a790-949526095f2f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:42.435517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8ed11-5a04-475c-a3db-c92232245865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:42.436236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8ed11-5a04-475c-a3db-c92232245865 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.436472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8ed11-5a04-475c-a3db-c92232245865 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.436750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8ed11-5a04-475c-a3db-c92232245865 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.481741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:42.484341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:42.500249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60946e8e-d4f0-4ae0-b785-ff09cc3525b6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:42.513143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8ed11-5a04-475c-a3db-c92232245865 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:42.514655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8ed11-5a04-475c-a3db-c92232245865 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:42.533133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36495c1a-29b6-4b30-a391-46004634a8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:42.533935 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36495c1a-29b6-4b30-a391-46004634a8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.534275 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36495c1a-29b6-4b30-a391-46004634a8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.534553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36495c1a-29b6-4b30-a391-46004634a8e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.536042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36495c1a-29b6-4b30-a391-46004634a8e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:42.537490 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36495c1a-29b6-4b30-a391-46004634a8e1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:42.539161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 41/79] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:42.539918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36495c1a-29b6-4b30-a391-46004634a8e1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:42.617026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 39/80] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:21:42.617026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8ed11-5a04-475c-a3db-c92232245865 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:42.641938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36495c1a-29b6-4b30-a391-46004634a8e1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:42.676469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf745a-ffb9-4f8c-acd0-0360f25c6252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:42.677383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf745a-ffb9-4f8c-acd0-0360f25c6252 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.677669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf745a-ffb9-4f8c-acd0-0360f25c6252 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.677947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf745a-ffb9-4f8c-acd0-0360f25c6252 None admin] PATH_INFO: `/v3/users/c145cd66ea9b4d3a85c89c24f585acff` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.679189 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf745a-ffb9-4f8c-acd0-0360f25c6252 None admin] RBAC: Authorizing `identity:get_user(user_id=c145cd66ea9b4d3a85c89c24f585acff)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:42.680668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faf745a-ffb9-4f8c-acd0-0360f25c6252 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:42.682311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 42/81] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:42 2022] GET /identity/v3/users/c145cd66ea9b4d3a85c89c24f585acff => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:42.683088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf745a-ffb9-4f8c-acd0-0360f25c6252 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:42.683403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:42.704247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:42.726448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8ed11-5a04-475c-a3db-c92232245865 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:42.760264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0dc792-f16d-4581-aac3-c585f40c4f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:42.761092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0dc792-f16d-4581-aac3-c585f40c4f4c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.761365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0dc792-f16d-4581-aac3-c585f40c4f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.761643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0dc792-f16d-4581-aac3-c585f40c4f4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.772389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0dc792-f16d-4581-aac3-c585f40c4f4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-4b0dc792-f16d-4581-aac3-c585f40c4f4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:42.780026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:42.780912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:42.781829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 40/82] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:42.782609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0dc792-f16d-4581-aac3-c585f40c4f4c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:42.789389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf745a-ffb9-4f8c-acd0-0360f25c6252 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:42.823936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a803e-e355-43e0-b7a4-5060ff05f901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:42.824812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a803e-e355-43e0-b7a4-5060ff05f901 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.825088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a803e-e355-43e0-b7a4-5060ff05f901 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.825358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a803e-e355-43e0-b7a4-5060ff05f901 None admin] PATH_INFO: `/v3/users/c2a06e0093704ec0865c012a4c2427dc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.826570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a803e-e355-43e0-b7a4-5060ff05f901 None admin] RBAC: Authorizing `identity:get_user(user_id=c2a06e0093704ec0865c012a4c2427dc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:42.828094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0a803e-e355-43e0-b7a4-5060ff05f901 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:42.829727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 43/83] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:42 2022] GET /identity/v3/users/c2a06e0093704ec0865c012a4c2427dc => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:42.830482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a803e-e355-43e0-b7a4-5060ff05f901 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:42.898262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0dc792-f16d-4581-aac3-c585f40c4f4c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:42.919199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:42.928674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:42.935083 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796f9c4-629d-404f-b77f-e2bd8a68aba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:42.935627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a803e-e355-43e0-b7a4-5060ff05f901 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:42.935754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796f9c4-629d-404f-b77f-e2bd8a68aba0 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.936035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796f9c4-629d-404f-b77f-e2bd8a68aba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.936325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796f9c4-629d-404f-b77f-e2bd8a68aba0 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.937079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796f9c4-629d-404f-b77f-e2bd8a68aba0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:42.938547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796f9c4-629d-404f-b77f-e2bd8a68aba0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:42.959298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 41/84] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:21:42 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:21:42.960043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796f9c4-629d-404f-b77f-e2bd8a68aba0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:42.974129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca106a02-87c8-4615-b138-8338ac7e2df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:42.975066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca106a02-87c8-4615-b138-8338ac7e2df2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:42.975350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca106a02-87c8-4615-b138-8338ac7e2df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:42.975625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca106a02-87c8-4615-b138-8338ac7e2df2 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:42.977062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca106a02-87c8-4615-b138-8338ac7e2df2 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:42.978598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca106a02-87c8-4615-b138-8338ac7e2df2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:42.980458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 44/85] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:42 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:42.983865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca106a02-87c8-4615-b138-8338ac7e2df2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:43.096492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796f9c4-629d-404f-b77f-e2bd8a68aba0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:43.114711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca106a02-87c8-4615-b138-8338ac7e2df2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:43.137131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294d016-5e91-4aa9-8c87-efedd7de4287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:43.137811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294d016-5e91-4aa9-8c87-efedd7de4287 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:43.139130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294d016-5e91-4aa9-8c87-efedd7de4287 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:43.139130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294d016-5e91-4aa9-8c87-efedd7de4287 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:43.156775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69c5ba-3f0f-4a7c-ba7f-6ccee6c020c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:43.157789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69c5ba-3f0f-4a7c-ba7f-6ccee6c020c2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:43.158251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69c5ba-3f0f-4a7c-ba7f-6ccee6c020c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:43.158712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69c5ba-3f0f-4a7c-ba7f-6ccee6c020c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:43.159651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69c5ba-3f0f-4a7c-ba7f-6ccee6c020c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:43.160030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69c5ba-3f0f-4a7c-ba7f-6ccee6c020c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:43.161441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69c5ba-3f0f-4a7c-ba7f-6ccee6c020c2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:43.170091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 45/86] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:21:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:43.171029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69c5ba-3f0f-4a7c-ba7f-6ccee6c020c2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:43.194593 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:43.206396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:43.224318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294d016-5e91-4aa9-8c87-efedd7de4287 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:43.226107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294d016-5e91-4aa9-8c87-efedd7de4287 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:43.302014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69c5ba-3f0f-4a7c-ba7f-6ccee6c020c2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:43.347995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83c62f-c82f-4ac7-8cf2-b382a1ab7996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:43.349024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83c62f-c82f-4ac7-8cf2-b382a1ab7996 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:43.349598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83c62f-c82f-4ac7-8cf2-b382a1ab7996 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:43.350090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83c62f-c82f-4ac7-8cf2-b382a1ab7996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:43.355085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 42/87] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:21:43.355818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294d016-5e91-4aa9-8c87-efedd7de4287 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:43.425436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83c62f-c82f-4ac7-8cf2-b382a1ab7996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:43.427785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83c62f-c82f-4ac7-8cf2-b382a1ab7996 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:43.446417 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:43.452897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:43.474779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294d016-5e91-4aa9-8c87-efedd7de4287 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:43.511413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98957e6b-e036-48da-a88c-29019f13cabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:43.512337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957e6b-e036-48da-a88c-29019f13cabc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:43.512626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957e6b-e036-48da-a88c-29019f13cabc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:43.512907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98957e6b-e036-48da-a88c-29019f13cabc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:43.513973 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98957e6b-e036-48da-a88c-29019f13cabc None admin] RBAC: Adding query filter params (role.id=b0d377da1a7d4260be1f7ae0715fda91, scope.project.id=314e1f70a6da40a79963abc77a1351c4, user.id=c2a06e0093704ec0865c012a4c2427dc) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:43.514090 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98957e6b-e036-48da-a88c-29019f13cabc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:43.515578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98957e6b-e036-48da-a88c-29019f13cabc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:43.522405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 43/88] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:43 2022] GET /identity/v3/role_assignments?user.id=c2a06e0093704ec0865c012a4c2427dc&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:43.523263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98957e6b-e036-48da-a88c-29019f13cabc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:43.542651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 46/89] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:21:43.543959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83c62f-c82f-4ac7-8cf2-b382a1ab7996 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:43.655602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83c62f-c82f-4ac7-8cf2-b382a1ab7996 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:43.675352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:43.696191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb7023-57e9-4fa0-9500-75beb733385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:43.697461 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb7023-57e9-4fa0-9500-75beb733385f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:43.698037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb7023-57e9-4fa0-9500-75beb733385f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:43.698522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb7023-57e9-4fa0-9500-75beb733385f None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:43.700177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb7023-57e9-4fa0-9500-75beb733385f None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:43.701894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb7023-57e9-4fa0-9500-75beb733385f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:43.704073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 47/90] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:43 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:43.705409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb7023-57e9-4fa0-9500-75beb733385f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:43.727066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98957e6b-e036-48da-a88c-29019f13cabc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:43.821819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb7023-57e9-4fa0-9500-75beb733385f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:43.835980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89be310-9ce8-4bb6-ba53-33819fb90aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:43.836888 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89be310-9ce8-4bb6-ba53-33819fb90aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:43.837171 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89be310-9ce8-4bb6-ba53-33819fb90aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:43.837449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89be310-9ce8-4bb6-ba53-33819fb90aa2 None admin] PATH_INFO: `/v3/users/c2a06e0093704ec0865c012a4c2427dc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:43.843411 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89be310-9ce8-4bb6-ba53-33819fb90aa2 None admin] RBAC: Authorizing `identity:get_user(user_id=c2a06e0093704ec0865c012a4c2427dc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:43.844878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89be310-9ce8-4bb6-ba53-33819fb90aa2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:43.854716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 44/91] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:43 2022] GET /identity/v3/users/c2a06e0093704ec0865c012a4c2427dc => generated 275 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:43.854716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89be310-9ce8-4bb6-ba53-33819fb90aa2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:43.876842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc7b4-5837-4d6d-958b-aa4c3b5b54f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:43.877718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc7b4-5837-4d6d-958b-aa4c3b5b54f8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:43.877999 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc7b4-5837-4d6d-958b-aa4c3b5b54f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:43.878272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc7b4-5837-4d6d-958b-aa4c3b5b54f8 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:43.879789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bc7b4-5837-4d6d-958b-aa4c3b5b54f8 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:43.883240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bc7b4-5837-4d6d-958b-aa4c3b5b54f8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:43.888081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 48/92] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:43 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:43.888622 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:43.888907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc7b4-5837-4d6d-958b-aa4c3b5b54f8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:43.980272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89be310-9ce8-4bb6-ba53-33819fb90aa2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:43.995852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:44.022779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211430f-e47a-4b02-8974-b9e69e7c12ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:44.023505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211430f-e47a-4b02-8974-b9e69e7c12ac None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:44.023799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211430f-e47a-4b02-8974-b9e69e7c12ac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:44.024086 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211430f-e47a-4b02-8974-b9e69e7c12ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:44.024939 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211430f-e47a-4b02-8974-b9e69e7c12ac None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=314e1f70a6da40a79963abc77a1351c4, user.id=c145cd66ea9b4d3a85c89c24f585acff) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:44.025259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211430f-e47a-4b02-8974-b9e69e7c12ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:44.026992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a211430f-e47a-4b02-8974-b9e69e7c12ac None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:44.033114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 45/93] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:43 2022] GET /identity/v3/role_assignments?user.id=c145cd66ea9b4d3a85c89c24f585acff&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:44.033993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211430f-e47a-4b02-8974-b9e69e7c12ac None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:44.135747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc7b4-5837-4d6d-958b-aa4c3b5b54f8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:44.147183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:44.169197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211430f-e47a-4b02-8974-b9e69e7c12ac None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:44.210865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5d9bf-b189-426f-9a41-c4ecbe27bf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:44.211555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5d9bf-b189-426f-9a41-c4ecbe27bf56 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:44.211845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5d9bf-b189-426f-9a41-c4ecbe27bf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:44.212128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5d9bf-b189-426f-9a41-c4ecbe27bf56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:44.261590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6942b4f-5d9d-4ca0-91d0-708a78591d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:44.263397 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6942b4f-5d9d-4ca0-91d0-708a78591d0f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:44.263709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6942b4f-5d9d-4ca0-91d0-708a78591d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:44.263994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6942b4f-5d9d-4ca0-91d0-708a78591d0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:44.270901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6942b4f-5d9d-4ca0-91d0-708a78591d0f None admin] RBAC: Adding query filter params (role.id=3531cd7306964cc5967f92c559890ae5, scope.project.id=314e1f70a6da40a79963abc77a1351c4, user.id=c2a06e0093704ec0865c012a4c2427dc) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:44.271238 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6942b4f-5d9d-4ca0-91d0-708a78591d0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:44.275528 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6942b4f-5d9d-4ca0-91d0-708a78591d0f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:44.292412 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 46/94] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:44 2022] GET /identity/v3/role_assignments?user.id=c2a06e0093704ec0865c012a4c2427dc&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=3531cd7306964cc5967f92c559890ae5 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:44.363180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5d9bf-b189-426f-9a41-c4ecbe27bf56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:44.364716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5d9bf-b189-426f-9a41-c4ecbe27bf56 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:44.438789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:44.585807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 49/95] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:21:44.596240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6942b4f-5d9d-4ca0-91d0-708a78591d0f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:44.788915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6942b4f-5d9d-4ca0-91d0-708a78591d0f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:44.890483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92553eba-54af-41e9-aab0-74869457681c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:44.891325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92553eba-54af-41e9-aab0-74869457681c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:44.891613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92553eba-54af-41e9-aab0-74869457681c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:44.891895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92553eba-54af-41e9-aab0-74869457681c None admin] PATH_INFO: `/v3/projects/76bc52f769d848d792eab0645e2cd4a1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:44.896960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92553eba-54af-41e9-aab0-74869457681c None admin] RBAC: Authorizing `identity:get_project(project_id=76bc52f769d848d792eab0645e2cd4a1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:44.898435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92553eba-54af-41e9-aab0-74869457681c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:44.903844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 47/96] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:44 2022] GET /identity/v3/projects/76bc52f769d848d792eab0645e2cd4a1 => generated 312 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:44.923902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5d9bf-b189-426f-9a41-c4ecbe27bf56 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:45.110034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5d9bf-b189-426f-9a41-c4ecbe27bf56 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:45.152477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c85eb-38be-4afa-be5b-64f4079bfae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:45.153477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c85eb-38be-4afa-be5b-64f4079bfae7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:45.153948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c85eb-38be-4afa-be5b-64f4079bfae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:45.154425 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c85eb-38be-4afa-be5b-64f4079bfae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:45.155494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c85eb-38be-4afa-be5b-64f4079bfae7 None admin] RBAC: Adding query filter params (role.id=b0d377da1a7d4260be1f7ae0715fda91, scope.project.id=76bc52f769d848d792eab0645e2cd4a1, user.id=c2a06e0093704ec0865c012a4c2427dc) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:45.155998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c85eb-38be-4afa-be5b-64f4079bfae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:45.157634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c85eb-38be-4afa-be5b-64f4079bfae7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:45.169704 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 50/97] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:44 2022] GET /identity/v3/role_assignments?user.id=c2a06e0093704ec0865c012a4c2427dc&scope.project.id=76bc52f769d848d792eab0645e2cd4a1&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:45.741587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d52361-1e98-4a03-a3e2-0b098129d78a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:45.742576 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d52361-1e98-4a03-a3e2-0b098129d78a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:45.743079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d52361-1e98-4a03-a3e2-0b098129d78a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:45.744658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 48/98] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:45.750096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f711875-21e5-4c06-8efe-fe17302b8529 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:45.750964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f711875-21e5-4c06-8efe-fe17302b8529 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:45.751434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f711875-21e5-4c06-8efe-fe17302b8529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:45.768394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3f711875-21e5-4c06-8efe-fe17302b8529 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:45.769924 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3f711875-21e5-4c06-8efe-fe17302b8529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:45.781272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:45.808873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebb014-f282-4c2b-9737-425a6b05db14 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:45.809180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebb014-f282-4c2b-9737-425a6b05db14 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:45.809428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebb014-f282-4c2b-9737-425a6b05db14 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:45.810979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 49/99] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:21:45.820462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e606f-de57-4de7-b243-fe3429c116e9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:45.820751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e606f-de57-4de7-b243-fe3429c116e9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:45.821036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e606f-de57-4de7-b243-fe3429c116e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:45.843121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b85e606f-de57-4de7-b243-fe3429c116e9 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:45.844323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b85e606f-de57-4de7-b243-fe3429c116e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:45.862015 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:45.876454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 51/100] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:45.913056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f711875-21e5-4c06-8efe-fe17302b8529 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:45.989442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 50/101] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:21:46.025884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e606f-de57-4de7-b243-fe3429c116e9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:46.097494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f711875-21e5-4c06-8efe-fe17302b8529 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:46.163480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7ce90-0121-4ed2-8e5e-a18ef8093324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:46.164303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7ce90-0121-4ed2-8e5e-a18ef8093324 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:46.164588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7ce90-0121-4ed2-8e5e-a18ef8093324 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:46.164868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7ce90-0121-4ed2-8e5e-a18ef8093324 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:46.167625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7ce90-0121-4ed2-8e5e-a18ef8093324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:46.169143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7ce90-0121-4ed2-8e5e-a18ef8093324 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:46.170904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 52/102] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:46.177110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7ce90-0121-4ed2-8e5e-a18ef8093324 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:46.208675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e606f-de57-4de7-b243-fe3429c116e9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:46.209580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:46.275325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc583c1-75f5-4e28-9e5e-559cda90f20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:46.276206 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc583c1-75f5-4e28-9e5e-559cda90f20c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:46.276430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc583c1-75f5-4e28-9e5e-559cda90f20c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:46.276708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc583c1-75f5-4e28-9e5e-559cda90f20c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:46.296464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc583c1-75f5-4e28-9e5e-559cda90f20c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-fcc583c1-75f5-4e28-9e5e-559cda90f20c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:46.305480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:46.306329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:21:46.307098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:46.307909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 51/103] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:21:46.313050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc583c1-75f5-4e28-9e5e-559cda90f20c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:46.326971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:46.370668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7ce90-0121-4ed2-8e5e-a18ef8093324 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:46.438866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b54fc-acc5-4816-95fb-0dd0c9c325b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:46.439864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b54fc-acc5-4816-95fb-0dd0c9c325b9 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:46.440283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b54fc-acc5-4816-95fb-0dd0c9c325b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:46.440691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b54fc-acc5-4816-95fb-0dd0c9c325b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:46.442021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b54fc-acc5-4816-95fb-0dd0c9c325b9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:46.447930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b54fc-acc5-4816-95fb-0dd0c9c325b9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:46.468336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc583c1-75f5-4e28-9e5e-559cda90f20c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:46.505633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 53/104] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:21:46 2022] POST /identity/v3/projects => generated 302 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:21:46.506823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91575f-631d-4cff-a547-b361978fc531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:46.507765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91575f-631d-4cff-a547-b361978fc531 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:46.508213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91575f-631d-4cff-a547-b361978fc531 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:46.508688 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91575f-631d-4cff-a547-b361978fc531 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:46.509679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91575f-631d-4cff-a547-b361978fc531 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:46.510161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91575f-631d-4cff-a547-b361978fc531 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:46.511791 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91575f-631d-4cff-a547-b361978fc531 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:46.522651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 52/105] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:21:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:46.528781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b54fc-acc5-4816-95fb-0dd0c9c325b9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:46.581003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d9912-8b7f-458b-ad29-a771e32e6b41 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:46.581513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d9912-8b7f-458b-ad29-a771e32e6b41 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:46.582300 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d9912-8b7f-458b-ad29-a771e32e6b41 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:46.584028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 53/106] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:21:46.589979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a678589-8ea6-4eb5-97fc-0829e744bc9d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:46.590535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a678589-8ea6-4eb5-97fc-0829e744bc9d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:46.591047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a678589-8ea6-4eb5-97fc-0829e744bc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:46.609022 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7a678589-8ea6-4eb5-97fc-0829e744bc9d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:46.610612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7a678589-8ea6-4eb5-97fc-0829e744bc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:46.637999 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:46.688658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b54fc-acc5-4816-95fb-0dd0c9c325b9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:46.715673 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:46.728364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 54/107] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:21:46.747899 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f6bb7-90c2-4f84-8c22-c33baaebb886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:46.748794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f6bb7-90c2-4f84-8c22-c33baaebb886 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:46.749077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f6bb7-90c2-4f84-8c22-c33baaebb886 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:46.749355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f6bb7-90c2-4f84-8c22-c33baaebb886 None admin] PATH_INFO: `/v3/users/cfa683300edc4b23931347ccc8d24e24` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:46.771095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f6bb7-90c2-4f84-8c22-c33baaebb886 None admin] RBAC: Authorizing `identity:get_user(user_id=cfa683300edc4b23931347ccc8d24e24)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:46.772714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f6bb7-90c2-4f84-8c22-c33baaebb886 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:46.773680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a678589-8ea6-4eb5-97fc-0829e744bc9d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:46.777802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 54/108] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:46 2022] GET /identity/v3/users/cfa683300edc4b23931347ccc8d24e24 => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:46.784318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f6bb7-90c2-4f84-8c22-c33baaebb886 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:46.913703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a678589-8ea6-4eb5-97fc-0829e744bc9d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:46.984343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed303fa3-f8e8-482e-93b4-9060f2e87fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:46.986328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed303fa3-f8e8-482e-93b4-9060f2e87fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:46.989781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f6bb7-90c2-4f84-8c22-c33baaebb886 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:46.990267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed303fa3-f8e8-482e-93b4-9060f2e87fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:46.990737 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed303fa3-f8e8-482e-93b4-9060f2e87fb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:47.012920 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed303fa3-f8e8-482e-93b4-9060f2e87fb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ed303fa3-f8e8-482e-93b4-9060f2e87fb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:47.031587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:47.032473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:47.033213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:47.037485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 55/109] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:47.044206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:47.054599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed303fa3-f8e8-482e-93b4-9060f2e87fb1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:47.069856 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83edbd8-d02d-4446-a54a-885d36ddffc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:47.070769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83edbd8-d02d-4446-a54a-885d36ddffc8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:47.071186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83edbd8-d02d-4446-a54a-885d36ddffc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:47.071615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83edbd8-d02d-4446-a54a-885d36ddffc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:47.144857 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83edbd8-d02d-4446-a54a-885d36ddffc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:47.146603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83edbd8-d02d-4446-a54a-885d36ddffc8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:47.168076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:47.258331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed303fa3-f8e8-482e-93b4-9060f2e87fb1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:47.324060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51763615-7dc6-4897-b8ff-65725e5af8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:47.325128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51763615-7dc6-4897-b8ff-65725e5af8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:47.325560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51763615-7dc6-4897-b8ff-65725e5af8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:47.325972 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51763615-7dc6-4897-b8ff-65725e5af8a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:47.327030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51763615-7dc6-4897-b8ff-65725e5af8a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:47.327480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51763615-7dc6-4897-b8ff-65725e5af8a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:47.329282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51763615-7dc6-4897-b8ff-65725e5af8a6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:47.342271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 56/110] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:21:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:47.348911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51763615-7dc6-4897-b8ff-65725e5af8a6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:47.431595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 55/111] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 08:21:47.435443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a2ec-19f1-47af-b002-596376c2bc52 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:47.435740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a2ec-19f1-47af-b002-596376c2bc52 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:47.436027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a2ec-19f1-47af-b002-596376c2bc52 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:47.437435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 56/112] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:47.438190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925a2ec-19f1-47af-b002-596376c2bc52 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:47.498019 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51763615-7dc6-4897-b8ff-65725e5af8a6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:47.507564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:47.534303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:47.564159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114f9ab-38dc-4b57-a185-004a0486e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:47.565426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f9ab-38dc-4b57-a185-004a0486e459 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:47.565931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f9ab-38dc-4b57-a185-004a0486e459 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:47.566407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114f9ab-38dc-4b57-a185-004a0486e459 None admin] PATH_INFO: `/v3/users/c2a06e0093704ec0865c012a4c2427dc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:47.568013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114f9ab-38dc-4b57-a185-004a0486e459 None admin] RBAC: Authorizing `identity:get_user(user_id=c2a06e0093704ec0865c012a4c2427dc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:47.570226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114f9ab-38dc-4b57-a185-004a0486e459 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:47.572270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 57/113] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:47 2022] GET /identity/v3/users/c2a06e0093704ec0865c012a4c2427dc => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:47.579154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052059b-9f73-49dd-ab17-20b4dd109c9a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:47.580546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052059b-9f73-49dd-ab17-20b4dd109c9a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:47.581834 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052059b-9f73-49dd-ab17-20b4dd109c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:47.613929 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8052059b-9f73-49dd-ab17-20b4dd109c9a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:47.616898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8052059b-9f73-49dd-ab17-20b4dd109c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:47.649784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925a2ec-19f1-47af-b002-596376c2bc52 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:47.694362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf82d68-c6aa-41ca-ba0f-33d3d9ce01e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:47.695527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf82d68-c6aa-41ca-ba0f-33d3d9ce01e9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:47.696534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf82d68-c6aa-41ca-ba0f-33d3d9ce01e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:47.696987 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf82d68-c6aa-41ca-ba0f-33d3d9ce01e9 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:47.698863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf82d68-c6aa-41ca-ba0f-33d3d9ce01e9 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:47.701222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf82d68-c6aa-41ca-ba0f-33d3d9ce01e9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:47.703905 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 57/114] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:47 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:47.705407 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf82d68-c6aa-41ca-ba0f-33d3d9ce01e9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:47.732668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 58/115] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:47.740766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dcafa-adcb-44d7-8fec-fac6e5675686 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:47.740766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dcafa-adcb-44d7-8fec-fac6e5675686 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:47.740766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dcafa-adcb-44d7-8fec-fac6e5675686 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:47.740766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 59/116] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:47.741797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dcafa-adcb-44d7-8fec-fac6e5675686 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:47.829515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:47.841711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf82d68-c6aa-41ca-ba0f-33d3d9ce01e9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:47.871277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dcafa-adcb-44d7-8fec-fac6e5675686 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:47.885847 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abde26-3bca-4f0a-9362-255cb1ef3fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:47.887267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abde26-3bca-4f0a-9362-255cb1ef3fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:47.888141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abde26-3bca-4f0a-9362-255cb1ef3fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:47.888559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abde26-3bca-4f0a-9362-255cb1ef3fe8 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:47.890546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:47.890805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abde26-3bca-4f0a-9362-255cb1ef3fe8 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:47.893130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abde26-3bca-4f0a-9362-255cb1ef3fe8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:47.895614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 58/117] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:47 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:47.898279 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0931c-d474-4590-a93a-5bbc8d0dc25e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:47.898856 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0931c-d474-4590-a93a-5bbc8d0dc25e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:47.899367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0931c-d474-4590-a93a-5bbc8d0dc25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:47.914005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e336-24b6-48b2-a8bd-0611fd750995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:47.914707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e336-24b6-48b2-a8bd-0611fd750995 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:47.915023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e336-24b6-48b2-a8bd-0611fd750995 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:47.915306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145e336-24b6-48b2-a8bd-0611fd750995 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:47.916168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145e336-24b6-48b2-a8bd-0611fd750995 None admin] RBAC: Adding query filter params (role.id=9a50f15db31f490294f370186ce2d7ff, scope.project.id=314e1f70a6da40a79963abc77a1351c4, user.id=cfa683300edc4b23931347ccc8d24e24) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:47.916491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145e336-24b6-48b2-a8bd-0611fd750995 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:47.917957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145e336-24b6-48b2-a8bd-0611fd750995 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:47.920300 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c0e0931c-d474-4590-a93a-5bbc8d0dc25e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:47.922280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c0e0931c-d474-4590-a93a-5bbc8d0dc25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:47.931110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 60/118] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:47 2022] GET /identity/v3/role_assignments?user.id=cfa683300edc4b23931347ccc8d24e24&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=9a50f15db31f490294f370186ce2d7ff => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:47.932133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e336-24b6-48b2-a8bd-0611fd750995 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:48.007874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 59/119] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:48.008715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0931c-d474-4590-a93a-5bbc8d0dc25e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:48.068093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145e336-24b6-48b2-a8bd-0611fd750995 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:48.106404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5199fc6-2868-4ba9-98fe-f3dafd522ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:48.107571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5199fc6-2868-4ba9-98fe-f3dafd522ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:48.108010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5199fc6-2868-4ba9-98fe-f3dafd522ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:48.108423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5199fc6-2868-4ba9-98fe-f3dafd522ec6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:48.111378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:48.120919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5199fc6-2868-4ba9-98fe-f3dafd522ec6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:48.123224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0931c-d474-4590-a93a-5bbc8d0dc25e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f5199fc6-2868-4ba9-98fe-f3dafd522ec6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:48.129770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:48.130857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:21:48.131748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:48.134049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 61/120] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:21:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:48.135523 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c051b8-d478-45ab-a401-bcfa70afc52f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:48.135965 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c051b8-d478-45ab-a401-bcfa70afc52f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:48.136376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c051b8-d478-45ab-a401-bcfa70afc52f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:48.137783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 62/121] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:48.138583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c051b8-d478-45ab-a401-bcfa70afc52f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:48.172362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745263d8-f038-4460-9403-56bd2504dd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:48.173344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745263d8-f038-4460-9403-56bd2504dd90 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:48.173632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745263d8-f038-4460-9403-56bd2504dd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:48.173909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745263d8-f038-4460-9403-56bd2504dd90 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4/users/c2a06e0093704ec0865c012a4c2427dc/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:48.177241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745263d8-f038-4460-9403-56bd2504dd90 None admin] RBAC: Authorizing `identity:create_grant(project_id=314e1f70a6da40a79963abc77a1351c4, user_id=c2a06e0093704ec0865c012a4c2427dc, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:48.179185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745263d8-f038-4460-9403-56bd2504dd90 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:48.195707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:48.206879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 60/122] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:21:48 2022] PUT /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4/users/c2a06e0093704ec0865c012a4c2427dc/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:21:48.208282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745263d8-f038-4460-9403-56bd2504dd90 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:48.342420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c051b8-d478-45ab-a401-bcfa70afc52f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:48.427421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1623b-6753-4177-82e4-05cc49a059f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:48.428221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1623b-6753-4177-82e4-05cc49a059f9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:48.429311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1623b-6753-4177-82e4-05cc49a059f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:48.429594 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1623b-6753-4177-82e4-05cc49a059f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:48.440165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745263d8-f038-4460-9403-56bd2504dd90 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:48.452659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd1623b-6753-4177-82e4-05cc49a059f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6bd1623b-6753-4177-82e4-05cc49a059f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:48.461010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:48.461881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:48.462617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:48.464060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 63/123] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:21:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:48.465489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0988e-48c3-4532-9b42-c7aea316fea9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:48.465740 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0988e-48c3-4532-9b42-c7aea316fea9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:48.466027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0988e-48c3-4532-9b42-c7aea316fea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:48.485987 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b40fa2-504d-4229-957b-05859d9d1319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:48.487114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b40fa2-504d-4229-957b-05859d9d1319 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:48.487590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b40fa2-504d-4229-957b-05859d9d1319 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:48.488105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b40fa2-504d-4229-957b-05859d9d1319 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:48.488955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a9f0988e-48c3-4532-9b42-c7aea316fea9 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:48.489314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b40fa2-504d-4229-957b-05859d9d1319 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:48.489807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b40fa2-504d-4229-957b-05859d9d1319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:48.490216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a9f0988e-48c3-4532-9b42-c7aea316fea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:48.491930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b40fa2-504d-4229-957b-05859d9d1319 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:48.501272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 61/124] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:21:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:48.502228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b40fa2-504d-4229-957b-05859d9d1319 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:48.541195 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:48.608713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 64/125] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:48.609603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0988e-48c3-4532-9b42-c7aea316fea9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:48.675090 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:48.715985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b40fa2-504d-4229-957b-05859d9d1319 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:48.782260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031524bb-502b-4670-b372-3d5fc376c135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:48.782970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031524bb-502b-4670-b372-3d5fc376c135 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:48.783257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031524bb-502b-4670-b372-3d5fc376c135 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:48.783535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031524bb-502b-4670-b372-3d5fc376c135 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:48.784297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031524bb-502b-4670-b372-3d5fc376c135 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:48.784606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031524bb-502b-4670-b372-3d5fc376c135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:48.786241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031524bb-502b-4670-b372-3d5fc376c135 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:48.803688 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0988e-48c3-4532-9b42-c7aea316fea9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:48.818306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 62/126] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:48.818953 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031524bb-502b-4670-b372-3d5fc376c135 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:48.850490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50c753-10af-4651-b824-add3f22c05f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:48.851475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c753-10af-4651-b824-add3f22c05f2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:48.851798 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c753-10af-4651-b824-add3f22c05f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:48.852079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50c753-10af-4651-b824-add3f22c05f2 None admin] PATH_INFO: `/v3/users/c2a06e0093704ec0865c012a4c2427dc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:48.854103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50c753-10af-4651-b824-add3f22c05f2 None admin] RBAC: Authorizing `identity:get_user(user_id=c2a06e0093704ec0865c012a4c2427dc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:48.856003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50c753-10af-4651-b824-add3f22c05f2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:48.858173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 65/127] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:48 2022] GET /identity/v3/users/c2a06e0093704ec0865c012a4c2427dc => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:48.859062 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50c753-10af-4651-b824-add3f22c05f2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:48.967906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:48.971999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50c753-10af-4651-b824-add3f22c05f2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:48.997974 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031524bb-502b-4670-b372-3d5fc376c135 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:49.011389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938054fa-497a-4e85-9d1a-91c0858a189f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:49.012369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938054fa-497a-4e85-9d1a-91c0858a189f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:49.012656 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938054fa-497a-4e85-9d1a-91c0858a189f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:49.012933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938054fa-497a-4e85-9d1a-91c0858a189f None admin] PATH_INFO: `/v3/users/c145cd66ea9b4d3a85c89c24f585acff` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:49.014520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938054fa-497a-4e85-9d1a-91c0858a189f None admin] RBAC: Authorizing `identity:get_user(user_id=c145cd66ea9b4d3a85c89c24f585acff)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:49.016135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938054fa-497a-4e85-9d1a-91c0858a189f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:49.018029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 66/128] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:48 2022] GET /identity/v3/users/c145cd66ea9b4d3a85c89c24f585acff => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:49.018641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938054fa-497a-4e85-9d1a-91c0858a189f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:49.029053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:49.068756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bda12-1048-4d93-8a68-96a7de37eeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:49.070340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bda12-1048-4d93-8a68-96a7de37eeb2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:49.070566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bda12-1048-4d93-8a68-96a7de37eeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:49.071066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bda12-1048-4d93-8a68-96a7de37eeb2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:49.111717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bda12-1048-4d93-8a68-96a7de37eeb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-700bda12-1048-4d93-8a68-96a7de37eeb2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:49.121071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:49.121942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:49.122678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:21:49.131196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 63/129] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:49.132124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bda12-1048-4d93-8a68-96a7de37eeb2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:49.138929 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938054fa-497a-4e85-9d1a-91c0858a189f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:49.177566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084c987-f902-4cca-947a-919969f77315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:49.178937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084c987-f902-4cca-947a-919969f77315 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:49.179415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084c987-f902-4cca-947a-919969f77315 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:49.179952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084c987-f902-4cca-947a-919969f77315 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:49.186171 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084c987-f902-4cca-947a-919969f77315 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:49.188018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084c987-f902-4cca-947a-919969f77315 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:49.194743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 67/130] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:49 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:49.195912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084c987-f902-4cca-947a-919969f77315 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:49.267174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:49.291517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bda12-1048-4d93-8a68-96a7de37eeb2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:49.329220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0ef3d-b501-4d8c-b511-73906aaf81be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:49.330477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0ef3d-b501-4d8c-b511-73906aaf81be None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:49.330982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0ef3d-b501-4d8c-b511-73906aaf81be None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:49.331445 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0ef3d-b501-4d8c-b511-73906aaf81be None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:49.333159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0ef3d-b501-4d8c-b511-73906aaf81be None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:49.334970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0ef3d-b501-4d8c-b511-73906aaf81be None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:49.337277 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 64/131] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:49 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:49.338731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0ef3d-b501-4d8c-b511-73906aaf81be None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:49.342978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:49.441991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084c987-f902-4cca-947a-919969f77315 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:49.493272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e5946-f34b-4fa4-9df6-0afe937c29fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:49.494110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e5946-f34b-4fa4-9df6-0afe937c29fc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:49.494400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e5946-f34b-4fa4-9df6-0afe937c29fc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:49.494674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e5946-f34b-4fa4-9df6-0afe937c29fc None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:49.495468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e5946-f34b-4fa4-9df6-0afe937c29fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:49.495775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e5946-f34b-4fa4-9df6-0afe937c29fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:49.497162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7e5946-f34b-4fa4-9df6-0afe937c29fc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:49.506966 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 68/132] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:21:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:49.507801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e5946-f34b-4fa4-9df6-0afe937c29fc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:49.521266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0ef3d-b501-4d8c-b511-73906aaf81be None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:49.556281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc6a22-9c4d-41ad-8322-0fc2fd79d0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:49.557456 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc6a22-9c4d-41ad-8322-0fc2fd79d0e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:49.557891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc6a22-9c4d-41ad-8322-0fc2fd79d0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:49.558314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc6a22-9c4d-41ad-8322-0fc2fd79d0e8 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4/users/c2a06e0093704ec0865c012a4c2427dc/roles/3531cd7306964cc5967f92c559890ae5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:49.560755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc6a22-9c4d-41ad-8322-0fc2fd79d0e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=314e1f70a6da40a79963abc77a1351c4, user_id=c2a06e0093704ec0865c012a4c2427dc, role_id=3531cd7306964cc5967f92c559890ae5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:49.562440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc6a22-9c4d-41ad-8322-0fc2fd79d0e8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:49.582861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 65/133] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:21:49 2022] PUT /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4/users/c2a06e0093704ec0865c012a4c2427dc/roles/3531cd7306964cc5967f92c559890ae5 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:21:49.585536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936e812-0e33-447c-a9a3-3b08eedfec83 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:49.586021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936e812-0e33-447c-a9a3-3b08eedfec83 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:49.586619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936e812-0e33-447c-a9a3-3b08eedfec83 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:49.589719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 66/134] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:49.595797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936e812-0e33-447c-a9a3-3b08eedfec83 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:49.673505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:49.673729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:49.689746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e5946-f34b-4fa4-9df6-0afe937c29fc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:49.723109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936e812-0e33-447c-a9a3-3b08eedfec83 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:49.768093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8258d81-691e-477b-b1ba-4fd908d2fdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:49.769471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8258d81-691e-477b-b1ba-4fd908d2fdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:49.769971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8258d81-691e-477b-b1ba-4fd908d2fdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:49.770448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8258d81-691e-477b-b1ba-4fd908d2fdf4 None admin] PATH_INFO: `/v3/users/c2a06e0093704ec0865c012a4c2427dc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:49.772087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8258d81-691e-477b-b1ba-4fd908d2fdf4 None admin] RBAC: Authorizing `identity:get_user(user_id=c2a06e0093704ec0865c012a4c2427dc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:49.773791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8258d81-691e-477b-b1ba-4fd908d2fdf4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:49.779473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 67/135] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:49 2022] GET /identity/v3/users/c2a06e0093704ec0865c012a4c2427dc => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:49.781038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ee712-fdcd-49c1-84d9-11f232408c4e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:49.781560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ee712-fdcd-49c1-84d9-11f232408c4e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:49.782038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ee712-fdcd-49c1-84d9-11f232408c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:49.800343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f47ee712-fdcd-49c1-84d9-11f232408c4e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:49.801965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f47ee712-fdcd-49c1-84d9-11f232408c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:49.816070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a270f8-6f32-4327-84ae-43922d9cbba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:49.817017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a270f8-6f32-4327-84ae-43922d9cbba6 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:49.817298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a270f8-6f32-4327-84ae-43922d9cbba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:49.817574 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a270f8-6f32-4327-84ae-43922d9cbba6 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4/users/c145cd66ea9b4d3a85c89c24f585acff/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:49.827492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a270f8-6f32-4327-84ae-43922d9cbba6 None admin] RBAC: Authorizing `identity:create_grant(project_id=314e1f70a6da40a79963abc77a1351c4, user_id=c145cd66ea9b4d3a85c89c24f585acff, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:49.828990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a270f8-6f32-4327-84ae-43922d9cbba6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:49.861260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 69/136] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:21:49 2022] PUT /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4/users/c145cd66ea9b4d3a85c89c24f585acff/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:21:49.862110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a270f8-6f32-4327-84ae-43922d9cbba6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:49.890630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 68/137] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:49.968390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ee712-fdcd-49c1-84d9-11f232408c4e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:50.068912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a270f8-6f32-4327-84ae-43922d9cbba6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:50.092467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:50.133172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42656986-2a84-47d3-a1ca-92e2cac718c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:50.134459 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42656986-2a84-47d3-a1ca-92e2cac718c7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:50.134967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42656986-2a84-47d3-a1ca-92e2cac718c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:50.135453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42656986-2a84-47d3-a1ca-92e2cac718c7 None admin] PATH_INFO: `/v3/projects/76bc52f769d848d792eab0645e2cd4a1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:50.136856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42656986-2a84-47d3-a1ca-92e2cac718c7 None admin] RBAC: Authorizing `identity:get_project(project_id=76bc52f769d848d792eab0645e2cd4a1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:50.138508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42656986-2a84-47d3-a1ca-92e2cac718c7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:50.140656 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 70/138] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:49 2022] GET /identity/v3/projects/76bc52f769d848d792eab0645e2cd4a1 => generated 312 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:50.147190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42656986-2a84-47d3-a1ca-92e2cac718c7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:50.182724 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ee712-fdcd-49c1-84d9-11f232408c4e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:50.188632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:50.237103 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ae64c-ac06-4d29-b884-465e762cdb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:50.238121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ae64c-ac06-4d29-b884-465e762cdb03 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:50.238882 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ae64c-ac06-4d29-b884-465e762cdb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:50.238882 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ae64c-ac06-4d29-b884-465e762cdb03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:50.240065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ae64c-ac06-4d29-b884-465e762cdb03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:50.241544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ae64c-ac06-4d29-b884-465e762cdb03 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:50.244780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 69/139] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:50.250971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ae64c-ac06-4d29-b884-465e762cdb03 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:50.354193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ae64c-ac06-4d29-b884-465e762cdb03 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:50.354962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42656986-2a84-47d3-a1ca-92e2cac718c7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:50.382515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:50.389995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e434c-cd56-4e27-a75c-641a1871df90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:50.390678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e434c-cd56-4e27-a75c-641a1871df90 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:50.391121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e434c-cd56-4e27-a75c-641a1871df90 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:50.391407 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e434c-cd56-4e27-a75c-641a1871df90 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:50.392159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e434c-cd56-4e27-a75c-641a1871df90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:50.393635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e434c-cd56-4e27-a75c-641a1871df90 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:50.416799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061c152-91bf-4a08-8171-5100d1e84f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:50.417721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c152-91bf-4a08-8171-5100d1e84f23 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:50.417999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c152-91bf-4a08-8171-5100d1e84f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:50.418279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061c152-91bf-4a08-8171-5100d1e84f23 None admin] PATH_INFO: `/v3/projects/76bc52f769d848d792eab0645e2cd4a1/users/c2a06e0093704ec0865c012a4c2427dc/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:50.422746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061c152-91bf-4a08-8171-5100d1e84f23 None admin] RBAC: Authorizing `identity:create_grant(project_id=76bc52f769d848d792eab0645e2cd4a1, user_id=c2a06e0093704ec0865c012a4c2427dc, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:50.422746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061c152-91bf-4a08-8171-5100d1e84f23 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:50.431816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 70/140] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:21:50 2022] POST /identity/v3/users => generated 283 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:21:50.446888 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 71/141] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:21:50 2022] PUT /identity/v3/projects/76bc52f769d848d792eab0645e2cd4a1/users/c2a06e0093704ec0865c012a4c2427dc/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:21:50.526682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6036d7-dc2b-4118-ba4a-a45a277567fb None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:50.527374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6036d7-dc2b-4118-ba4a-a45a277567fb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:50.527852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6036d7-dc2b-4118-ba4a-a45a277567fb None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:50.529694 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 71/142] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:50.539808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292f1aa-ff5b-4c4a-b15f-191e2a6960e6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:50.540115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292f1aa-ff5b-4c4a-b15f-191e2a6960e6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:50.540375 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292f1aa-ff5b-4c4a-b15f-191e2a6960e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:50.563495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c292f1aa-ff5b-4c4a-b15f-191e2a6960e6 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:50.565070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c292f1aa-ff5b-4c4a-b15f-191e2a6960e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:50.661770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 72/143] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:50.698169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6036d7-dc2b-4118-ba4a-a45a277567fb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:50.825770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6036d7-dc2b-4118-ba4a-a45a277567fb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:50.861627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451101a7-f34d-4197-9114-100190321544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:50.862498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451101a7-f34d-4197-9114-100190321544 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:50.862881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451101a7-f34d-4197-9114-100190321544 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:50.863171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451101a7-f34d-4197-9114-100190321544 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:50.874731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451101a7-f34d-4197-9114-100190321544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-451101a7-f34d-4197-9114-100190321544 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:50.882823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:50.883706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:21:50.884666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 72/144] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:50.890781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292f1aa-ff5b-4c4a-b15f-191e2a6960e6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:50.915135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:51.029133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292f1aa-ff5b-4c4a-b15f-191e2a6960e6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:51.071818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d7ede-3aea-42c2-a1e9-42f27c52996a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:51.072560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d7ede-3aea-42c2-a1e9-42f27c52996a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:51.072790 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d7ede-3aea-42c2-a1e9-42f27c52996a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:51.073068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d7ede-3aea-42c2-a1e9-42f27c52996a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:51.073846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d7ede-3aea-42c2-a1e9-42f27c52996a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:51.074153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d7ede-3aea-42c2-a1e9-42f27c52996a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:51.075578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d7ede-3aea-42c2-a1e9-42f27c52996a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:51.089249 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 73/145] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:21:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:51.094574 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451101a7-f34d-4197-9114-100190321544 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:51.152230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:51.217144 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451101a7-f34d-4197-9114-100190321544 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:51.255173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1ef61-eb2f-4461-9e92-08a5a527845e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:51.256068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1ef61-eb2f-4461-9e92-08a5a527845e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:51.256353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1ef61-eb2f-4461-9e92-08a5a527845e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:51.256629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1ef61-eb2f-4461-9e92-08a5a527845e None admin] PATH_INFO: `/v3/users/cfa683300edc4b23931347ccc8d24e24` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:51.257856 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1ef61-eb2f-4461-9e92-08a5a527845e None admin] RBAC: Authorizing `identity:get_user(user_id=cfa683300edc4b23931347ccc8d24e24)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:51.259437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1ef61-eb2f-4461-9e92-08a5a527845e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:51.261335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 73/146] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:51 2022] GET /identity/v3/users/cfa683300edc4b23931347ccc8d24e24 => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:51.267767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d7ede-3aea-42c2-a1e9-42f27c52996a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:51.372521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:51.376691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404db25-c8d4-4cbb-be17-0fc21c1ffcb4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:51.376982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404db25-c8d4-4cbb-be17-0fc21c1ffcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:51.377300 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404db25-c8d4-4cbb-be17-0fc21c1ffcb4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:51.379539 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 74/147] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:51.386532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f29df-7c70-4fb4-a2dc-c580514fce20 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:51.386845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f29df-7c70-4fb4-a2dc-c580514fce20 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:51.387128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f29df-7c70-4fb4-a2dc-c580514fce20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:51.399598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d7ede-3aea-42c2-a1e9-42f27c52996a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:51.411088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-677f29df-7c70-4fb4-a2dc-c580514fce20 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:51.412586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-677f29df-7c70-4fb4-a2dc-c580514fce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:51.452790 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16601df-5dcf-4408-93af-35bf89d20111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:51.453682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16601df-5dcf-4408-93af-35bf89d20111 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:51.453959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16601df-5dcf-4408-93af-35bf89d20111 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:51.454234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16601df-5dcf-4408-93af-35bf89d20111 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:51.455706 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16601df-5dcf-4408-93af-35bf89d20111 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:51.458728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16601df-5dcf-4408-93af-35bf89d20111 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:51.461715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 74/148] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:51 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:51.468770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16601df-5dcf-4408-93af-35bf89d20111 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:51.542550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 75/149] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:51.591137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f29df-7c70-4fb4-a2dc-c580514fce20 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:51.639139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16601df-5dcf-4408-93af-35bf89d20111 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:51.663923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:51.708851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52ec3d-43b7-4105-a716-75375516e950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:51.710659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52ec3d-43b7-4105-a716-75375516e950 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:51.710962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52ec3d-43b7-4105-a716-75375516e950 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:51.711235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52ec3d-43b7-4105-a716-75375516e950 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4/users/cfa683300edc4b23931347ccc8d24e24/roles/9a50f15db31f490294f370186ce2d7ff` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:51.717933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52ec3d-43b7-4105-a716-75375516e950 None admin] RBAC: Authorizing `identity:create_grant(project_id=314e1f70a6da40a79963abc77a1351c4, user_id=cfa683300edc4b23931347ccc8d24e24, role_id=9a50f15db31f490294f370186ce2d7ff)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:51.719494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52ec3d-43b7-4105-a716-75375516e950 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:51.744992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 75/150] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:21:51 2022] PUT /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4/users/cfa683300edc4b23931347ccc8d24e24/roles/9a50f15db31f490294f370186ce2d7ff => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:21:51.758993 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f29df-7c70-4fb4-a2dc-c580514fce20 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:51.796719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0e21e-618e-4e6e-833d-89ac658377d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:51.797584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0e21e-618e-4e6e-833d-89ac658377d3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:51.797868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0e21e-618e-4e6e-833d-89ac658377d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:51.798136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0e21e-618e-4e6e-833d-89ac658377d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:51.810272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0e21e-618e-4e6e-833d-89ac658377d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-93e0e21e-618e-4e6e-833d-89ac658377d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:51.818283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:51.819222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:21:51.820042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 76/151] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:51.826322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52ec3d-43b7-4105-a716-75375516e950 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:51.829895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:51.934235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52ec3d-43b7-4105-a716-75375516e950 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:51.973350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ae5f3-ac83-439b-a826-8c55f4f6cf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:51.973951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ae5f3-ac83-439b-a826-8c55f4f6cf63 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:51.974240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ae5f3-ac83-439b-a826-8c55f4f6cf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:51.974514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ae5f3-ac83-439b-a826-8c55f4f6cf63 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:51.975351 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ae5f3-ac83-439b-a826-8c55f4f6cf63 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:51.976526 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ae5f3-ac83-439b-a826-8c55f4f6cf63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:51.978048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ae5f3-ac83-439b-a826-8c55f4f6cf63 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:51.987506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 76/152] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:21:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:51.993383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0e21e-618e-4e6e-833d-89ac658377d3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:52.109460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0e21e-618e-4e6e-833d-89ac658377d3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:52.111049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:52.148214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3ed49-6e0d-41b1-b7fb-0f512b1e5dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:52.149154 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3ed49-6e0d-41b1-b7fb-0f512b1e5dde None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:52.149443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3ed49-6e0d-41b1-b7fb-0f512b1e5dde None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:52.149721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b3ed49-6e0d-41b1-b7fb-0f512b1e5dde None admin] PATH_INFO: `/v3/users/c2a06e0093704ec0865c012a4c2427dc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:52.151212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3ed49-6e0d-41b1-b7fb-0f512b1e5dde None admin] RBAC: Authorizing `identity:get_user(user_id=c2a06e0093704ec0865c012a4c2427dc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:52.153096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b3ed49-6e0d-41b1-b7fb-0f512b1e5dde None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:52.154605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 77/153] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:51 2022] GET /identity/v3/users/c2a06e0093704ec0865c012a4c2427dc => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:52.161369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ae5f3-ac83-439b-a826-8c55f4f6cf63 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:52.231401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:52.269079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ae5f3-ac83-439b-a826-8c55f4f6cf63 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:52.304357 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec98e31-777a-4dbf-9962-b3d9a1707619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:52.305022 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec98e31-777a-4dbf-9962-b3d9a1707619 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:52.305303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec98e31-777a-4dbf-9962-b3d9a1707619 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:52.305578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec98e31-777a-4dbf-9962-b3d9a1707619 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:52.375728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec98e31-777a-4dbf-9962-b3d9a1707619 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:52.377293 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec98e31-777a-4dbf-9962-b3d9a1707619 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:52.443245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:52.478603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 77/154] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:21:52.487946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3ed49-6e0d-41b1-b7fb-0f512b1e5dde None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:52.606629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b3ed49-6e0d-41b1-b7fb-0f512b1e5dde None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:52.649089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be941200-2550-44d2-a3fe-8f2cad0b5731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:52.649959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be941200-2550-44d2-a3fe-8f2cad0b5731 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:52.650238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be941200-2550-44d2-a3fe-8f2cad0b5731 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:52.650512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be941200-2550-44d2-a3fe-8f2cad0b5731 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:52.653291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be941200-2550-44d2-a3fe-8f2cad0b5731 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:52.654853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be941200-2550-44d2-a3fe-8f2cad0b5731 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:52.656620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 78/155] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:52 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:52.662753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec98e31-777a-4dbf-9962-b3d9a1707619 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:52.783345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec98e31-777a-4dbf-9962-b3d9a1707619 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:52.814463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dd115-bc34-4292-89bb-136fa3410d3d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:52.814804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dd115-bc34-4292-89bb-136fa3410d3d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:52.815136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dd115-bc34-4292-89bb-136fa3410d3d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:52.817044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 79/156] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:52.823370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add0b43-e58d-4e0c-863a-dbf3c5f0ea05 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:52.823654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add0b43-e58d-4e0c-863a-dbf3c5f0ea05 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:52.823932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add0b43-e58d-4e0c-863a-dbf3c5f0ea05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:52.843783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343d84f-054d-4a05-814f-7c5b8a907493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:52.845580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343d84f-054d-4a05-814f-7c5b8a907493 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:52.845866 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343d84f-054d-4a05-814f-7c5b8a907493 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:52.846139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d343d84f-054d-4a05-814f-7c5b8a907493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:52.847016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343d84f-054d-4a05-814f-7c5b8a907493 None admin] RBAC: Adding query filter params (role.id=b0d377da1a7d4260be1f7ae0715fda91, scope.project.id=314e1f70a6da40a79963abc77a1351c4, user.id=c2a06e0093704ec0865c012a4c2427dc) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:52.847372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343d84f-054d-4a05-814f-7c5b8a907493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:52.848846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d343d84f-054d-4a05-814f-7c5b8a907493 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:52.857493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0add0b43-e58d-4e0c-863a-dbf3c5f0ea05 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:52.858715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0add0b43-e58d-4e0c-863a-dbf3c5f0ea05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:52.861927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 78/157] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:52 2022] GET /identity/v3/role_assignments?user.id=c2a06e0093704ec0865c012a4c2427dc&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:52.921151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:52.978814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 80/158] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:53.019794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343d84f-054d-4a05-814f-7c5b8a907493 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:53.072666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:53.168655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d343d84f-054d-4a05-814f-7c5b8a907493 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:53.208841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86816e-11bf-437c-be02-e36cd6848af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:53.209689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86816e-11bf-437c-be02-e36cd6848af7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:53.209976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86816e-11bf-437c-be02-e36cd6848af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:53.210252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86816e-11bf-437c-be02-e36cd6848af7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:53.225525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86816e-11bf-437c-be02-e36cd6848af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-7d86816e-11bf-437c-be02-e36cd6848af7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:53.233816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:53.234792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:21:53.235602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:53.238793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 79/159] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:21:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:53.243350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add0b43-e58d-4e0c-863a-dbf3c5f0ea05 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:53.382064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add0b43-e58d-4e0c-863a-dbf3c5f0ea05 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:53.424556 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb506cd-6ad7-4958-9f4b-4b0c33e38d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:53.425318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb506cd-6ad7-4958-9f4b-4b0c33e38d2a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:53.425606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb506cd-6ad7-4958-9f4b-4b0c33e38d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:53.425877 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb506cd-6ad7-4958-9f4b-4b0c33e38d2a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:53.426632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb506cd-6ad7-4958-9f4b-4b0c33e38d2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:53.426971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb506cd-6ad7-4958-9f4b-4b0c33e38d2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:53.428359 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb506cd-6ad7-4958-9f4b-4b0c33e38d2a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:53.442006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 81/160] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:21:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:53.447836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86816e-11bf-437c-be02-e36cd6848af7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:53.448862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc552228-d305-416a-a371-65b39c8ff19f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:53.449145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc552228-d305-416a-a371-65b39c8ff19f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:53.449421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc552228-d305-416a-a371-65b39c8ff19f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:53.450975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 82/161] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:53.474591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a4630-4186-43bc-902e-866af7810900 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:53.474927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a4630-4186-43bc-902e-866af7810900 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:53.475171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a4630-4186-43bc-902e-866af7810900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:53.501321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e64a4630-4186-43bc-902e-866af7810900 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:53.502790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e64a4630-4186-43bc-902e-866af7810900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:53.547486 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:53.616946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 83/162] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:21:53.617329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:53.639053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86816e-11bf-437c-be02-e36cd6848af7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:53.656063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a4630-4186-43bc-902e-866af7810900 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:53.702829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6174baf-3b2b-46cb-b081-1f1208e36d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:53.703735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6174baf-3b2b-46cb-b081-1f1208e36d20 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:53.704020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6174baf-3b2b-46cb-b081-1f1208e36d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:53.704295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6174baf-3b2b-46cb-b081-1f1208e36d20 None admin] PATH_INFO: `/v3/users/c2a06e0093704ec0865c012a4c2427dc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:53.708116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6174baf-3b2b-46cb-b081-1f1208e36d20 None admin] RBAC: Authorizing `identity:get_user(user_id=c2a06e0093704ec0865c012a4c2427dc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:53.709778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6174baf-3b2b-46cb-b081-1f1208e36d20 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:53.711749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 80/163] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:53 2022] GET /identity/v3/users/c2a06e0093704ec0865c012a4c2427dc => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:53.721932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6174baf-3b2b-46cb-b081-1f1208e36d20 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:53.863258 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a4630-4186-43bc-902e-866af7810900 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:53.902086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59019de-feb9-469d-99e7-d4fcc9381138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:53.903278 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59019de-feb9-469d-99e7-d4fcc9381138 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:53.903716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59019de-feb9-469d-99e7-d4fcc9381138 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:53.904134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59019de-feb9-469d-99e7-d4fcc9381138 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:53.917747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59019de-feb9-469d-99e7-d4fcc9381138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e59019de-feb9-469d-99e7-d4fcc9381138 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:53.925804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:53.926898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:53.928038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:21:53.931631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 84/164] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:21:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:21:53.943722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59019de-feb9-469d-99e7-d4fcc9381138 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:53.949197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6174baf-3b2b-46cb-b081-1f1208e36d20 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:53.979244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:54.031408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1c048-7969-4ee5-ae5a-31f37f4cea2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:54.032328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1c048-7969-4ee5-ae5a-31f37f4cea2a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:54.032769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1c048-7969-4ee5-ae5a-31f37f4cea2a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:54.033229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1c048-7969-4ee5-ae5a-31f37f4cea2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:54.069474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59019de-feb9-469d-99e7-d4fcc9381138 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:54.089324 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:54.106075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55147329-47be-43e1-a189-a89dceddde47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:54.107060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55147329-47be-43e1-a189-a89dceddde47 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:54.107569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55147329-47be-43e1-a189-a89dceddde47 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:54.108031 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55147329-47be-43e1-a189-a89dceddde47 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:54.108997 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55147329-47be-43e1-a189-a89dceddde47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:54.109458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55147329-47be-43e1-a189-a89dceddde47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:54.111025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55147329-47be-43e1-a189-a89dceddde47 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:54.120310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 85/165] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:54.125939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55147329-47be-43e1-a189-a89dceddde47 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:54.191451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1c048-7969-4ee5-ae5a-31f37f4cea2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:54.192846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1c048-7969-4ee5-ae5a-31f37f4cea2a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:54.291920 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:54.315864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55147329-47be-43e1-a189-a89dceddde47 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:54.338599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 81/166] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 08:21:54.339966 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458edad2-78e9-448f-9a8c-f53f76c14a43 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:54.340254 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458edad2-78e9-448f-9a8c-f53f76c14a43 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:54.340529 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458edad2-78e9-448f-9a8c-f53f76c14a43 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:54.341795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 82/167] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:54.347347 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458edad2-78e9-448f-9a8c-f53f76c14a43 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:54.381573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65b301-9ded-494c-a990-c885be8732af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:54.382633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65b301-9ded-494c-a990-c885be8732af None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:54.383042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65b301-9ded-494c-a990-c885be8732af None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:54.383408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65b301-9ded-494c-a990-c885be8732af None admin] PATH_INFO: `/v3/users/c145cd66ea9b4d3a85c89c24f585acff` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:54.385025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65b301-9ded-494c-a990-c885be8732af None admin] RBAC: Authorizing `identity:get_user(user_id=c145cd66ea9b4d3a85c89c24f585acff)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:54.387144 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65b301-9ded-494c-a990-c885be8732af None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:54.389093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 86/168] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:54 2022] GET /identity/v3/users/c145cd66ea9b4d3a85c89c24f585acff => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:54.390363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201c6f-289d-4f5e-9148-de54405f47d4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:54.390646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201c6f-289d-4f5e-9148-de54405f47d4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:54.390962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84201c6f-289d-4f5e-9148-de54405f47d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:54.409384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-84201c6f-289d-4f5e-9148-de54405f47d4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:54.410649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-84201c6f-289d-4f5e-9148-de54405f47d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:54.472643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:54.533192 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 87/169] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:54.534000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84201c6f-289d-4f5e-9148-de54405f47d4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:54.547441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458edad2-78e9-448f-9a8c-f53f76c14a43 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:54.613803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bbcbd-427c-4c32-9e12-e30bd29b301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:54.614812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bbcbd-427c-4c32-9e12-e30bd29b301b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:54.615098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bbcbd-427c-4c32-9e12-e30bd29b301b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:54.615373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bbcbd-427c-4c32-9e12-e30bd29b301b None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:54.619423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bbcbd-427c-4c32-9e12-e30bd29b301b None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:54.620909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bbcbd-427c-4c32-9e12-e30bd29b301b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:54.622980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 83/170] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:54 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:54.623763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bbcbd-427c-4c32-9e12-e30bd29b301b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:54.651221 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:54.678851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84201c6f-289d-4f5e-9148-de54405f47d4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:54.735495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d507c1-da4b-444b-aaff-649f0f26cd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:54.736110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d507c1-da4b-444b-aaff-649f0f26cd93 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:54.736366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d507c1-da4b-444b-aaff-649f0f26cd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:54.736636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d507c1-da4b-444b-aaff-649f0f26cd93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:54.866339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d507c1-da4b-444b-aaff-649f0f26cd93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:54.867855 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d507c1-da4b-444b-aaff-649f0f26cd93 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:54.869583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bbcbd-427c-4c32-9e12-e30bd29b301b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:54.936277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:54.959457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09179fa7-1112-419d-ad01-6e3b397817bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:54.960298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09179fa7-1112-419d-ad01-6e3b397817bc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:54.960590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09179fa7-1112-419d-ad01-6e3b397817bc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:54.960864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09179fa7-1112-419d-ad01-6e3b397817bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:54.997891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09179fa7-1112-419d-ad01-6e3b397817bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-09179fa7-1112-419d-ad01-6e3b397817bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:55.005722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:55.006565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:55.007450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:55.015257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 84/171] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:55.015950 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09179fa7-1112-419d-ad01-6e3b397817bc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:55.028816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:55.064327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 88/172] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:21:55.065573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d507c1-da4b-444b-aaff-649f0f26cd93 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:55.223661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d507c1-da4b-444b-aaff-649f0f26cd93 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:55.226902 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09179fa7-1112-419d-ad01-6e3b397817bc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:55.283208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d020d-0ccb-4d10-8bc7-f2ed3d9b413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:55.283805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d020d-0ccb-4d10-8bc7-f2ed3d9b413d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.284089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d020d-0ccb-4d10-8bc7-f2ed3d9b413d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.284360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d020d-0ccb-4d10-8bc7-f2ed3d9b413d None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.286025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d020d-0ccb-4d10-8bc7-f2ed3d9b413d None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:55.286337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d020d-0ccb-4d10-8bc7-f2ed3d9b413d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:55.287780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d020d-0ccb-4d10-8bc7-f2ed3d9b413d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:55.291057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6f335-5a33-4259-a7aa-26df87121a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:55.291729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6f335-5a33-4259-a7aa-26df87121a9d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.292007 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6f335-5a33-4259-a7aa-26df87121a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.292279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6f335-5a33-4259-a7aa-26df87121a9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.293135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6f335-5a33-4259-a7aa-26df87121a9d None admin] RBAC: Adding query filter params (role.id=3531cd7306964cc5967f92c559890ae5, scope.project.id=314e1f70a6da40a79963abc77a1351c4, user.id=c2a06e0093704ec0865c012a4c2427dc) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:55.293456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6f335-5a33-4259-a7aa-26df87121a9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:55.294958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6f335-5a33-4259-a7aa-26df87121a9d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:55.307865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 89/173] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:21:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:55.308734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d020d-0ccb-4d10-8bc7-f2ed3d9b413d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:55.309373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 85/174] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:55 2022] GET /identity/v3/role_assignments?user.id=c2a06e0093704ec0865c012a4c2427dc&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=3531cd7306964cc5967f92c559890ae5 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:55.310849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73b2a6-f346-4918-b802-905e10675867 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.311140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73b2a6-f346-4918-b802-905e10675867 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.311418 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a73b2a6-f346-4918-b802-905e10675867 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.312810 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 86/175] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:55.316347 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73b2a6-f346-4918-b802-905e10675867 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:55.392820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:55.415085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:55.477227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a73b2a6-f346-4918-b802-905e10675867 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:55.515462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54a56b-6a34-493a-93ba-25ea17399b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:55.516413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54a56b-6a34-493a-93ba-25ea17399b25 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.516703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54a56b-6a34-493a-93ba-25ea17399b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.516983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54a56b-6a34-493a-93ba-25ea17399b25 None admin] PATH_INFO: `/v3/users/c2a06e0093704ec0865c012a4c2427dc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.518209 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54a56b-6a34-493a-93ba-25ea17399b25 None admin] RBAC: Authorizing `identity:get_user(user_id=c2a06e0093704ec0865c012a4c2427dc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:55.520267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54a56b-6a34-493a-93ba-25ea17399b25 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:55.521959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 87/176] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:55 2022] GET /identity/v3/users/c2a06e0093704ec0865c012a4c2427dc => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:55.526005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf1f38-8b62-4f1f-9086-f6a02fd9020e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.526284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf1f38-8b62-4f1f-9086-f6a02fd9020e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.526553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf1f38-8b62-4f1f-9086-f6a02fd9020e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.549388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-bfbf1f38-8b62-4f1f-9086-f6a02fd9020e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:55.550647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bfbf1f38-8b62-4f1f-9086-f6a02fd9020e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:55.563584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d020d-0ccb-4d10-8bc7-f2ed3d9b413d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:55.605277 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c6e9b-b2e7-48e8-80df-55fd4078b459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:55.606416 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c6e9b-b2e7-48e8-80df-55fd4078b459 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.606720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c6e9b-b2e7-48e8-80df-55fd4078b459 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.607226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c6e9b-b2e7-48e8-80df-55fd4078b459 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.608489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c6e9b-b2e7-48e8-80df-55fd4078b459 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:55.610329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c6e9b-b2e7-48e8-80df-55fd4078b459 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:55.612971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 90/177] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:55 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:55.614727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa07a4-7af9-45e6-8ed1-c74f4fab929c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.615037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa07a4-7af9-45e6-8ed1-c74f4fab929c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.615311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa07a4-7af9-45e6-8ed1-c74f4fab929c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.619748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 91/178] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:55.620494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa07a4-7af9-45e6-8ed1-c74f4fab929c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:55.638724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 88/179] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:55.639669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf1f38-8b62-4f1f-9086-f6a02fd9020e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:55.698166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:55.735018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa07a4-7af9-45e6-8ed1-c74f4fab929c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:55.745845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:55.752051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf1f38-8b62-4f1f-9086-f6a02fd9020e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:55.771535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c928a-b711-4ba6-8880-206208026021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:55.772182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c928a-b711-4ba6-8880-206208026021 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.772460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c928a-b711-4ba6-8880-206208026021 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.772759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c928a-b711-4ba6-8880-206208026021 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.786319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7c7ba-47f3-4a22-bed5-ea4131dcb286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:55.787013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7c7ba-47f3-4a22-bed5-ea4131dcb286 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.787307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7c7ba-47f3-4a22-bed5-ea4131dcb286 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.787566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7c7ba-47f3-4a22-bed5-ea4131dcb286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.788417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7c7ba-47f3-4a22-bed5-ea4131dcb286 None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=314e1f70a6da40a79963abc77a1351c4, user.id=c145cd66ea9b4d3a85c89c24f585acff) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:55.788736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7c7ba-47f3-4a22-bed5-ea4131dcb286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:55.790508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7c7ba-47f3-4a22-bed5-ea4131dcb286 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:55.796300 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 89/180] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:55 2022] GET /identity/v3/role_assignments?user.id=c145cd66ea9b4d3a85c89c24f585acff&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:55.797459 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c895e-a2d5-4b82-b520-acffdc117aec None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.797744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c895e-a2d5-4b82-b520-acffdc117aec None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.798012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c895e-a2d5-4b82-b520-acffdc117aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.828206 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-973c895e-a2d5-4b82-b520-acffdc117aec None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:55.829903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-973c895e-a2d5-4b82-b520-acffdc117aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:55.847041 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c928a-b711-4ba6-8880-206208026021 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:55.848943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c928a-b711-4ba6-8880-206208026021 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:55.904696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 90/181] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:55.905868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f55e-09e2-4106-98db-2ed9c5448f51 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.906157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f55e-09e2-4106-98db-2ed9c5448f51 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.906431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f55e-09e2-4106-98db-2ed9c5448f51 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.912251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 91/182] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:55.913159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435f55e-09e2-4106-98db-2ed9c5448f51 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:55.963770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:55.986617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 92/183] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:21:55.988762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cfc83-6878-4d89-9d8c-536fb0110bd3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.989053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cfc83-6878-4d89-9d8c-536fb0110bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.989334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cfc83-6878-4d89-9d8c-536fb0110bd3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:55.990626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 93/184] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:55.996423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e35d6-95a1-4933-aefe-01f8958c2916 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:55.996423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e35d6-95a1-4933-aefe-01f8958c2916 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:55.996423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e35d6-95a1-4933-aefe-01f8958c2916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.015496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7d3e35d6-95a1-4933-aefe-01f8958c2916 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:56.016705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7d3e35d6-95a1-4933-aefe-01f8958c2916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:56.021578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:56.033248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435f55e-09e2-4106-98db-2ed9c5448f51 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:56.067705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2005be9-4ce6-4416-a968-0dc217b4dab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:56.068504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2005be9-4ce6-4416-a968-0dc217b4dab4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.068789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2005be9-4ce6-4416-a968-0dc217b4dab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.069062 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2005be9-4ce6-4416-a968-0dc217b4dab4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.078916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2005be9-4ce6-4416-a968-0dc217b4dab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-c2005be9-4ce6-4416-a968-0dc217b4dab4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:56.086595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:56.087648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:56.088955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 92/185] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:21:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:56.090650 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2005be9-4ce6-4416-a968-0dc217b4dab4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.093436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 94/186] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:56.094346 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e35d6-95a1-4933-aefe-01f8958c2916 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.191989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e35d6-95a1-4933-aefe-01f8958c2916 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:56.199320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2005be9-4ce6-4416-a968-0dc217b4dab4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:56.200946 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:56.227215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e541e-93a7-48b1-aca7-33c09effdc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:56.228081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e541e-93a7-48b1-aca7-33c09effdc30 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.228354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e541e-93a7-48b1-aca7-33c09effdc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.228641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e541e-93a7-48b1-aca7-33c09effdc30 None admin] PATH_INFO: `/v3/projects/76bc52f769d848d792eab0645e2cd4a1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.229822 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e541e-93a7-48b1-aca7-33c09effdc30 None admin] RBAC: Authorizing `identity:get_project(project_id=76bc52f769d848d792eab0645e2cd4a1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:56.231336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e541e-93a7-48b1-aca7-33c09effdc30 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:56.233058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 95/187] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:56 2022] GET /identity/v3/projects/76bc52f769d848d792eab0645e2cd4a1 => generated 312 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:56.233896 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad262d-7b05-469b-a4e1-ebe5108136f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:56.234446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a456-d514-44e9-9639-3928908d53d1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.234446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a456-d514-44e9-9639-3928908d53d1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.234710 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad262d-7b05-469b-a4e1-ebe5108136f0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8a456-d514-44e9-9639-3928908d53d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.235012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad262d-7b05-469b-a4e1-ebe5108136f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.235284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad262d-7b05-469b-a4e1-ebe5108136f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.245901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad262d-7b05-469b-a4e1-ebe5108136f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:56.251743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-74d8a456-d514-44e9-9639-3928908d53d1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:56.252932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-74d8a456-d514-44e9-9639-3928908d53d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-33ad262d-7b05-469b-a4e1-ebe5108136f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:56.253933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:56.255064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:56.255827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 93/188] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:21:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:56.256715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad262d-7b05-469b-a4e1-ebe5108136f0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.257417 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:56.324263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 96/189] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:56.325800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8a456-d514-44e9-9639-3928908d53d1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.364772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad262d-7b05-469b-a4e1-ebe5108136f0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:56.398005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a240ae-83cd-41f9-b9b3-e19d61f11a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:56.398717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a240ae-83cd-41f9-b9b3-e19d61f11a64 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.399022 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a240ae-83cd-41f9-b9b3-e19d61f11a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.399300 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a240ae-83cd-41f9-b9b3-e19d61f11a64 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.400055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a240ae-83cd-41f9-b9b3-e19d61f11a64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:56.400364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a240ae-83cd-41f9-b9b3-e19d61f11a64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:56.401770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a240ae-83cd-41f9-b9b3-e19d61f11a64 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:56.409976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 94/190] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:56.410682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a240ae-83cd-41f9-b9b3-e19d61f11a64 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.429493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8a456-d514-44e9-9639-3928908d53d1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:56.458836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:56.464261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def417a7-b1eb-4281-8430-ee26ff12aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:56.465070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def417a7-b1eb-4281-8430-ee26ff12aa55 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.465355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def417a7-b1eb-4281-8430-ee26ff12aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.465625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def417a7-b1eb-4281-8430-ee26ff12aa55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.466826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def417a7-b1eb-4281-8430-ee26ff12aa55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:56.468299 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def417a7-b1eb-4281-8430-ee26ff12aa55 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:56.469959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 97/191] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:56.470713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def417a7-b1eb-4281-8430-ee26ff12aa55 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.481314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:56.534910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a240ae-83cd-41f9-b9b3-e19d61f11a64 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:56.567560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3c428-6ea9-4b44-bc9b-117b4777800a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:56.568212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3c428-6ea9-4b44-bc9b-117b4777800a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.568497 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3c428-6ea9-4b44-bc9b-117b4777800a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.568771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3c428-6ea9-4b44-bc9b-117b4777800a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.569607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3c428-6ea9-4b44-bc9b-117b4777800a None admin] RBAC: Adding query filter params (role.id=b0d377da1a7d4260be1f7ae0715fda91, scope.project.id=76bc52f769d848d792eab0645e2cd4a1, user.id=c2a06e0093704ec0865c012a4c2427dc) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:56.569915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3c428-6ea9-4b44-bc9b-117b4777800a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:56.571430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3c428-6ea9-4b44-bc9b-117b4777800a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:56.574900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def417a7-b1eb-4281-8430-ee26ff12aa55 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:56.577223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 95/192] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:56 2022] GET /identity/v3/role_assignments?user.id=c2a06e0093704ec0865c012a4c2427dc&scope.project.id=76bc52f769d848d792eab0645e2cd4a1&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:56.594982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3c428-6ea9-4b44-bc9b-117b4777800a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.615989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e9160-1e8b-4f84-bc45-ace0c6afadc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:56.616587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e9160-1e8b-4f84-bc45-ace0c6afadc7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.616870 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e9160-1e8b-4f84-bc45-ace0c6afadc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.617148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e9160-1e8b-4f84-bc45-ace0c6afadc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.617913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e9160-1e8b-4f84-bc45-ace0c6afadc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:56.618218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e9160-1e8b-4f84-bc45-ace0c6afadc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:56.619662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e9160-1e8b-4f84-bc45-ace0c6afadc7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:56.633169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 98/193] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:21:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:56.633169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e9160-1e8b-4f84-bc45-ace0c6afadc7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.691642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:56.705525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3c428-6ea9-4b44-bc9b-117b4777800a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:56.707805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:56.744166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e9160-1e8b-4f84-bc45-ace0c6afadc7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:56.744395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f91499-a46f-4d5c-bcd7-8bf6c74689aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:56.745126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f91499-a46f-4d5c-bcd7-8bf6c74689aa None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.745409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f91499-a46f-4d5c-bcd7-8bf6c74689aa None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.745702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f91499-a46f-4d5c-bcd7-8bf6c74689aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.759112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f91499-a46f-4d5c-bcd7-8bf6c74689aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a4f91499-a46f-4d5c-bcd7-8bf6c74689aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:56.767601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:56.768665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:21:56.769573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 96/194] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:56.775655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f91499-a46f-4d5c-bcd7-8bf6c74689aa None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.790196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f4f46-bc8f-4812-a944-369720c8f2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:56.791665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f4f46-bc8f-4812-a944-369720c8f2df None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.791665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f4f46-bc8f-4812-a944-369720c8f2df None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.791665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f4f46-bc8f-4812-a944-369720c8f2df None admin] PATH_INFO: `/v3/users/683477f356844d4e80bb88115e1d8de4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.806565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f4f46-bc8f-4812-a944-369720c8f2df None admin] RBAC: Authorizing `identity:get_user(user_id=683477f356844d4e80bb88115e1d8de4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:56.808056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344f4f46-bc8f-4812-a944-369720c8f2df None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:56.809783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 99/195] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:56 2022] GET /identity/v3/users/683477f356844d4e80bb88115e1d8de4 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:56.810544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f4f46-bc8f-4812-a944-369720c8f2df None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.890467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f91499-a46f-4d5c-bcd7-8bf6c74689aa None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:56.909352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f4f46-bc8f-4812-a944-369720c8f2df None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:56.924129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59999919-b16d-4cb0-8810-bf95a9eec30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:56.924805 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999919-b16d-4cb0-8810-bf95a9eec30e None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.925109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999919-b16d-4cb0-8810-bf95a9eec30e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.925395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59999919-b16d-4cb0-8810-bf95a9eec30e None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.926108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59999919-b16d-4cb0-8810-bf95a9eec30e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:56.927593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59999919-b16d-4cb0-8810-bf95a9eec30e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:56.934958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:56.945068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f100cd-fd15-4aca-b6bf-b535724147c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:56.945952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f100cd-fd15-4aca-b6bf-b535724147c8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:56.946240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f100cd-fd15-4aca-b6bf-b535724147c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:56.946514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f100cd-fd15-4aca-b6bf-b535724147c8 None admin] PATH_INFO: `/v3/users/683477f356844d4e80bb88115e1d8de4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:56.958957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 97/196] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:21:56 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:21:56.959156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f100cd-fd15-4aca-b6bf-b535724147c8 None admin] RBAC: Authorizing `identity:get_user(user_id=683477f356844d4e80bb88115e1d8de4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:56.959259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59999919-b16d-4cb0-8810-bf95a9eec30e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.959333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f100cd-fd15-4aca-b6bf-b535724147c8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:56.959333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 100/197] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:56 2022] GET /identity/v3/users/683477f356844d4e80bb88115e1d8de4 => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:56.959333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f100cd-fd15-4aca-b6bf-b535724147c8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:56.959500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:57.076634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59999919-b16d-4cb0-8810-bf95a9eec30e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:57.080930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f100cd-fd15-4aca-b6bf-b535724147c8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:57.114993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163d10a-3bfd-4bc7-bc16-3c1c5cf7e92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:57.115633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163d10a-3bfd-4bc7-bc16-3c1c5cf7e92e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:57.115917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163d10a-3bfd-4bc7-bc16-3c1c5cf7e92e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:57.116212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163d10a-3bfd-4bc7-bc16-3c1c5cf7e92e None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:57.116969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163d10a-3bfd-4bc7-bc16-3c1c5cf7e92e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:57.117284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163d10a-3bfd-4bc7-bc16-3c1c5cf7e92e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:57.118679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163d10a-3bfd-4bc7-bc16-3c1c5cf7e92e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:57.125225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59743a-72a9-45b7-aa8a-5a21e12d03fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:57.125225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59743a-72a9-45b7-aa8a-5a21e12d03fd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:57.126426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59743a-72a9-45b7-aa8a-5a21e12d03fd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:57.126426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59743a-72a9-45b7-aa8a-5a21e12d03fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:57.129211 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 98/198] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:21:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:57.129921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163d10a-3bfd-4bc7-bc16-3c1c5cf7e92e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:57.207893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:57.208062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:57.219773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59743a-72a9-45b7-aa8a-5a21e12d03fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:57.221297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59743a-72a9-45b7-aa8a-5a21e12d03fd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:57.254072 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163d10a-3bfd-4bc7-bc16-3c1c5cf7e92e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:57.307170 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be15ac2-5236-4cc9-a13e-12f14456b234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:57.307847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be15ac2-5236-4cc9-a13e-12f14456b234 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:57.308126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be15ac2-5236-4cc9-a13e-12f14456b234 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:57.308399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be15ac2-5236-4cc9-a13e-12f14456b234 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:57.351384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 101/199] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:21:57.353049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59743a-72a9-45b7-aa8a-5a21e12d03fd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:57.384270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be15ac2-5236-4cc9-a13e-12f14456b234 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:57.385789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be15ac2-5236-4cc9-a13e-12f14456b234 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:57.467088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:57.476292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59743a-72a9-45b7-aa8a-5a21e12d03fd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:57.484296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:57.518610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0eb540-bf97-4566-ba4b-7754b468f875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:57.519986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0eb540-bf97-4566-ba4b-7754b468f875 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:57.520446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0eb540-bf97-4566-ba4b-7754b468f875 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:57.520952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0eb540-bf97-4566-ba4b-7754b468f875 None admin] PATH_INFO: `/v3/users/cfa683300edc4b23931347ccc8d24e24` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:57.522378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0eb540-bf97-4566-ba4b-7754b468f875 None admin] RBAC: Authorizing `identity:get_user(user_id=cfa683300edc4b23931347ccc8d24e24)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:57.524361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0eb540-bf97-4566-ba4b-7754b468f875 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:57.530968 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 102/201] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:57 2022] GET /identity/v3/users/cfa683300edc4b23931347ccc8d24e24 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:57.532128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0eb540-bf97-4566-ba4b-7754b468f875 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:57.535585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 99/201] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:21:57.536394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be15ac2-5236-4cc9-a13e-12f14456b234 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:57.646885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0eb540-bf97-4566-ba4b-7754b468f875 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:57.648712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be15ac2-5236-4cc9-a13e-12f14456b234 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:57.683424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38840a-82a0-4d09-ad4a-fa7bf9106dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:57.684440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0dd2d-28a8-4d58-8e1d-7ee4f403a703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:57.684895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38840a-82a0-4d09-ad4a-fa7bf9106dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:57.685096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0dd2d-28a8-4d58-8e1d-7ee4f403a703 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:57.685378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0dd2d-28a8-4d58-8e1d-7ee4f403a703 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:57.685652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0dd2d-28a8-4d58-8e1d-7ee4f403a703 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:57.685941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38840a-82a0-4d09-ad4a-fa7bf9106dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:57.686415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38840a-82a0-4d09-ad4a-fa7bf9106dc1 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:57.687965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38840a-82a0-4d09-ad4a-fa7bf9106dc1 None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:57.689856 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38840a-82a0-4d09-ad4a-fa7bf9106dc1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:57.692045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 103/202] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:57 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:57.693137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38840a-82a0-4d09-ad4a-fa7bf9106dc1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:57.721704 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:57.728992 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:57.760998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0dd2d-28a8-4d58-8e1d-7ee4f403a703 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:57.762348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0dd2d-28a8-4d58-8e1d-7ee4f403a703 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:57.804061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38840a-82a0-4d09-ad4a-fa7bf9106dc1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:57.842242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5817-2c78-473e-902f-b9655cc57e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:57.843660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5817-2c78-473e-902f-b9655cc57e1b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:57.844138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5817-2c78-473e-902f-b9655cc57e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:57.844603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5817-2c78-473e-902f-b9655cc57e1b None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:57.846096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff5817-2c78-473e-902f-b9655cc57e1b None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:57.847929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff5817-2c78-473e-902f-b9655cc57e1b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:57.850334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 104/203] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:57 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:57.851481 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5817-2c78-473e-902f-b9655cc57e1b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:57.870428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 100/204] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:21:57.875104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0dd2d-28a8-4d58-8e1d-7ee4f403a703 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:57.939878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:57.960357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:57.964963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5817-2c78-473e-902f-b9655cc57e1b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:57.979337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0dd2d-28a8-4d58-8e1d-7ee4f403a703 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:58.001693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadef46-f0be-4f80-82a7-c0fa6065961b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:58.002643 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadef46-f0be-4f80-82a7-c0fa6065961b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:58.003276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadef46-f0be-4f80-82a7-c0fa6065961b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:58.003868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadef46-f0be-4f80-82a7-c0fa6065961b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:58.004918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadef46-f0be-4f80-82a7-c0fa6065961b None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=c862664ec560416a936d940af366502c, user.id=683477f356844d4e80bb88115e1d8de4) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:58.005425 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadef46-f0be-4f80-82a7-c0fa6065961b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:58.007341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadef46-f0be-4f80-82a7-c0fa6065961b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:58.013625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 105/205] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:57 2022] GET /identity/v3/role_assignments?user.id=683477f356844d4e80bb88115e1d8de4&scope.project.id=c862664ec560416a936d940af366502c&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:58.014601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f12c5-7b23-4b47-9bca-f94e4e0c890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:58.014942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadef46-f0be-4f80-82a7-c0fa6065961b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:58.015328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f12c5-7b23-4b47-9bca-f94e4e0c890a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:58.015632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f12c5-7b23-4b47-9bca-f94e4e0c890a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:58.015959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f12c5-7b23-4b47-9bca-f94e4e0c890a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:58.016816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f12c5-7b23-4b47-9bca-f94e4e0c890a None admin] RBAC: Adding query filter params (role.id=3531cd7306964cc5967f92c559890ae5, scope.project.id=c862664ec560416a936d940af366502c, user.id=683477f356844d4e80bb88115e1d8de4) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:58.017130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f12c5-7b23-4b47-9bca-f94e4e0c890a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:58.018876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f12c5-7b23-4b47-9bca-f94e4e0c890a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:58.025422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 101/206] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:57 2022] GET /identity/v3/role_assignments?user.id=683477f356844d4e80bb88115e1d8de4&scope.project.id=c862664ec560416a936d940af366502c&role.id=3531cd7306964cc5967f92c559890ae5 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:58.133604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadef46-f0be-4f80-82a7-c0fa6065961b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:58.179586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9416af-f86a-4691-b6d4-f33b5e24311b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:58.180508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9416af-f86a-4691-b6d4-f33b5e24311b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:58.180790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9416af-f86a-4691-b6d4-f33b5e24311b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:58.181060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9416af-f86a-4691-b6d4-f33b5e24311b None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:58.182460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9416af-f86a-4691-b6d4-f33b5e24311b None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:58.184007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9416af-f86a-4691-b6d4-f33b5e24311b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:58.185830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 106/207] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:21:58 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:58.192615 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f12c5-7b23-4b47-9bca-f94e4e0c890a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:58.349373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f12c5-7b23-4b47-9bca-f94e4e0c890a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:58.378522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:58.384841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25835e-c42d-4a97-804b-f3b0d1f374cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:58.385515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25835e-c42d-4a97-804b-f3b0d1f374cf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:58.385798 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25835e-c42d-4a97-804b-f3b0d1f374cf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:58.386324 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25835e-c42d-4a97-804b-f3b0d1f374cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:58.387226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25835e-c42d-4a97-804b-f3b0d1f374cf None admin] RBAC: Adding query filter params (role.id=9a50f15db31f490294f370186ce2d7ff, scope.project.id=314e1f70a6da40a79963abc77a1351c4, user.id=cfa683300edc4b23931347ccc8d24e24) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:58.387542 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25835e-c42d-4a97-804b-f3b0d1f374cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:58.389035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25835e-c42d-4a97-804b-f3b0d1f374cf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:58.398863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 102/208] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:21:58 2022] GET /identity/v3/role_assignments?user.id=cfa683300edc4b23931347ccc8d24e24&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=9a50f15db31f490294f370186ce2d7ff => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:59.191607 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa21238-8b41-4bdc-a5a9-089838882ad2 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:59.192182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa21238-8b41-4bdc-a5a9-089838882ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:59.192512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa21238-8b41-4bdc-a5a9-089838882ad2 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:59.194377 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 107/209] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:59.199746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6698-577c-4634-82d1-3ff72bc749d3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:59.200259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6698-577c-4634-82d1-3ff72bc749d3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:59.200742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6698-577c-4634-82d1-3ff72bc749d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:59.218344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a9fe6698-577c-4634-82d1-3ff72bc749d3 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:59.219843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a9fe6698-577c-4634-82d1-3ff72bc749d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:59.288138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 103/210] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:21:59.313084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa21238-8b41-4bdc-a5a9-089838882ad2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:59.325336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24663264-a291-4d74-a6ae-2c6527314fa0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:59.325857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24663264-a291-4d74-a6ae-2c6527314fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:59.326265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24663264-a291-4d74-a6ae-2c6527314fa0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:59.327898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 104/211] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:21:59.333937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e3f75-45c7-432e-8856-ed0608b6f0e4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:59.334458 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e3f75-45c7-432e-8856-ed0608b6f0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:59.334961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e3f75-45c7-432e-8856-ed0608b6f0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:59.349923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:59.353114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-798e3f75-45c7-432e-8856-ed0608b6f0e4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:21:59.354821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-798e3f75-45c7-432e-8856-ed0608b6f0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:21:59.433926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa21238-8b41-4bdc-a5a9-089838882ad2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:59.451232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 105/212] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:21:59.483799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e3f75-45c7-432e-8856-ed0608b6f0e4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:59.485661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77720eb1-7aa3-4b5c-840c-a1aeed3c1349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:59.486472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77720eb1-7aa3-4b5c-840c-a1aeed3c1349 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:59.486789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77720eb1-7aa3-4b5c-840c-a1aeed3c1349 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:59.487227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77720eb1-7aa3-4b5c-840c-a1aeed3c1349 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:59.500343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77720eb1-7aa3-4b5c-840c-a1aeed3c1349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-77720eb1-7aa3-4b5c-840c-a1aeed3c1349 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:21:59.508914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:21:59.509878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:21:59.510644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:21:59.513546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 108/213] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:21:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:21:59.526765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77720eb1-7aa3-4b5c-840c-a1aeed3c1349 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:59.558513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:59.618291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:59.620288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e3f75-45c7-432e-8856-ed0608b6f0e4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:59.629567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77720eb1-7aa3-4b5c-840c-a1aeed3c1349 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:59.664261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1e486-b7c3-4ead-a1a7-6bf4bab1e674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:59.664886 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7ef2b-18c3-45f7-a0d2-2dc526c8754c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:59.665068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e486-b7c3-4ead-a1a7-6bf4bab1e674 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:59.665354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e486-b7c3-4ead-a1a7-6bf4bab1e674 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:59.665630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1e486-b7c3-4ead-a1a7-6bf4bab1e674 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:59.665910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ef2b-18c3-45f7-a0d2-2dc526c8754c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:59.666277 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ef2b-18c3-45f7-a0d2-2dc526c8754c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:59.666634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ef2b-18c3-45f7-a0d2-2dc526c8754c None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:59.666829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1e486-b7c3-4ead-a1a7-6bf4bab1e674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:59.667615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7ef2b-18c3-45f7-a0d2-2dc526c8754c None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:21:59.668063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7ef2b-18c3-45f7-a0d2-2dc526c8754c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:59.668290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe1e486-b7c3-4ead-a1a7-6bf4bab1e674 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:59.669599 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a7ef2b-18c3-45f7-a0d2-2dc526c8754c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:59.670102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 106/214] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:21:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:59.675677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1e486-b7c3-4ead-a1a7-6bf4bab1e674 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:59.686952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 109/215] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:21:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:21:59.688795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7ef2b-18c3-45f7-a0d2-2dc526c8754c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:21:59.792284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1e486-b7c3-4ead-a1a7-6bf4bab1e674 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:59.809985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7ef2b-18c3-45f7-a0d2-2dc526c8754c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:21:59.816682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:21:59.830386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f3f81-237a-462c-b5e7-f41d1b272438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:59.831302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f3f81-237a-462c-b5e7-f41d1b272438 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:59.831694 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f3f81-237a-462c-b5e7-f41d1b272438 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:59.832063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f3f81-237a-462c-b5e7-f41d1b272438 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:59.832875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f3f81-237a-462c-b5e7-f41d1b272438 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:59.834408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f3f81-237a-462c-b5e7-f41d1b272438 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:59.853012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b60ad-bbe8-47f6-b034-097e9e0dc678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:21:59.854004 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b60ad-bbe8-47f6-b034-097e9e0dc678 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:21:59.854288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b60ad-bbe8-47f6-b034-097e9e0dc678 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:21:59.854609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b60ad-bbe8-47f6-b034-097e9e0dc678 None admin] PATH_INFO: `/v3/users/584a201ce6e74faab6f445189615b23d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:21:59.860469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 107/216] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:21:59 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:21:59.878087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b60ad-bbe8-47f6-b034-097e9e0dc678 None admin] RBAC: Authorizing `identity:get_user(user_id=584a201ce6e74faab6f445189615b23d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:21:59.880303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b60ad-bbe8-47f6-b034-097e9e0dc678 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:21:59.882723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 110/217] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:21:59 2022] GET /identity/v3/users/584a201ce6e74faab6f445189615b23d => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:21:59.888283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f3f81-237a-462c-b5e7-f41d1b272438 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:00.007305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f3f81-237a-462c-b5e7-f41d1b272438 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:00.044416 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12019ef3-32a2-4440-8bbe-83f99c07fba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:00.045012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12019ef3-32a2-4440-8bbe-83f99c07fba7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:00.045298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12019ef3-32a2-4440-8bbe-83f99c07fba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:00.045574 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12019ef3-32a2-4440-8bbe-83f99c07fba7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:00.093965 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:00.117837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12019ef3-32a2-4440-8bbe-83f99c07fba7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:00.119399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12019ef3-32a2-4440-8bbe-83f99c07fba7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:00.227015 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 108/218] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:21:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:00.232776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b60ad-bbe8-47f6-b034-097e9e0dc678 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:00.247606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:00.287097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3be22-cd32-4cf2-82e0-32775728b6c0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:00.287382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3be22-cd32-4cf2-82e0-32775728b6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:00.287868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb3be22-cd32-4cf2-82e0-32775728b6c0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:00.289446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 109/219] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:00.295326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90825f87-902e-4fd6-b5eb-6b49043b59a9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:00.295880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90825f87-902e-4fd6-b5eb-6b49043b59a9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:00.296368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90825f87-902e-4fd6-b5eb-6b49043b59a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:00.318532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-90825f87-902e-4fd6-b5eb-6b49043b59a9 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:00.320127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-90825f87-902e-4fd6-b5eb-6b49043b59a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:00.379152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b60ad-bbe8-47f6-b034-097e9e0dc678 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:00.393922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 110/220] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:00.396128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f0ecb-e5fc-4ca7-898a-976a5990335f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:00.396624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f0ecb-e5fc-4ca7-898a-976a5990335f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:00.397114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f0ecb-e5fc-4ca7-898a-976a5990335f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:00.398705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 111/221] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:00.404718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d0724-950a-4448-aad9-c93a48235363 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:00.405251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d0724-950a-4448-aad9-c93a48235363 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:00.405718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d0724-950a-4448-aad9-c93a48235363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:00.424163 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1834a1-c3bf-4e8a-95a0-0555d45e87eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:00.425136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1834a1-c3bf-4e8a-95a0-0555d45e87eb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:00.425588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1834a1-c3bf-4e8a-95a0-0555d45e87eb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:00.426029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1834a1-c3bf-4e8a-95a0-0555d45e87eb None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:00.427400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6d9d0724-950a-4448-aad9-c93a48235363 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:00.427513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1834a1-c3bf-4e8a-95a0-0555d45e87eb None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:00.428904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6d9d0724-950a-4448-aad9-c93a48235363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:00.429013 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1834a1-c3bf-4e8a-95a0-0555d45e87eb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:00.431057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 111/222] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:00 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:00.435597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:00.435746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1834a1-c3bf-4e8a-95a0-0555d45e87eb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:00.521580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 112/223] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:00.522801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d0724-950a-4448-aad9-c93a48235363 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:00.538309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:00.573820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1834a1-c3bf-4e8a-95a0-0555d45e87eb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:00.611408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049fcb1-00b5-44b7-8971-d3dc6e042ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:00.612222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049fcb1-00b5-44b7-8971-d3dc6e042ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:00.612496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049fcb1-00b5-44b7-8971-d3dc6e042ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:00.612990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049fcb1-00b5-44b7-8971-d3dc6e042ed9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:00.633423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049fcb1-00b5-44b7-8971-d3dc6e042ed9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a049fcb1-00b5-44b7-8971-d3dc6e042ed9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:00.647190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:00.648314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:00.649280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:00.652126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 112/224] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:22:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:00.652800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d0724-950a-4448-aad9-c93a48235363 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:00.653675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049fcb1-00b5-44b7-8971-d3dc6e042ed9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:00.693352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a297-d2ab-4c14-8c9d-5be34ffd2af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:00.693948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a297-d2ab-4c14-8c9d-5be34ffd2af3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:00.694232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a297-d2ab-4c14-8c9d-5be34ffd2af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:00.694508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03a297-d2ab-4c14-8c9d-5be34ffd2af3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:00.695385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03a297-d2ab-4c14-8c9d-5be34ffd2af3 None admin] RBAC: Adding query filter params (role.id=b0d377da1a7d4260be1f7ae0715fda91, scope.project.id=c862664ec560416a936d940af366502c, user.id=584a201ce6e74faab6f445189615b23d) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:00.695705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03a297-d2ab-4c14-8c9d-5be34ffd2af3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:00.697158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03a297-d2ab-4c14-8c9d-5be34ffd2af3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:00.703109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 113/225] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:00 2022] GET /identity/v3/role_assignments?user.id=584a201ce6e74faab6f445189615b23d&scope.project.id=c862664ec560416a936d940af366502c&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:00.710479 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a297-d2ab-4c14-8c9d-5be34ffd2af3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:00.714631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:00.843855 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049fcb1-00b5-44b7-8971-d3dc6e042ed9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:00.917877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03a297-d2ab-4c14-8c9d-5be34ffd2af3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:00.923649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f6c66-000f-4699-b139-c59363c23465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:00.924490 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f6c66-000f-4699-b139-c59363c23465 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:00.924779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f6c66-000f-4699-b139-c59363c23465 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:00.925393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f6c66-000f-4699-b139-c59363c23465 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:00.935224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:00.940533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f6c66-000f-4699-b139-c59363c23465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-eb0f6c66-000f-4699-b139-c59363c23465 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:00.950770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:00.951653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:00.952448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 113/226] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:22:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:00.958428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f6c66-000f-4699-b139-c59363c23465 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:00.961349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afb166-a863-4dbf-9944-315ca7f777ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:00.962280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb166-a863-4dbf-9944-315ca7f777ba None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:00.962790 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb166-a863-4dbf-9944-315ca7f777ba None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:00.963608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb166-a863-4dbf-9944-315ca7f777ba None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:00.964575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afb166-a863-4dbf-9944-315ca7f777ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:00.965076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afb166-a863-4dbf-9944-315ca7f777ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:00.966682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afb166-a863-4dbf-9944-315ca7f777ba None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:00.976660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 114/227] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:00.988011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afb166-a863-4dbf-9944-315ca7f777ba None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:01.058064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:01.064627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f6c66-000f-4699-b139-c59363c23465 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:01.106802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9cbc3-8f95-4ac9-8516-13298c687a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:01.106802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9cbc3-8f95-4ac9-8516-13298c687a3f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:01.106802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9cbc3-8f95-4ac9-8516-13298c687a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:01.106802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f9cbc3-8f95-4ac9-8516-13298c687a3f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:01.107212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9cbc3-8f95-4ac9-8516-13298c687a3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:01.107212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9cbc3-8f95-4ac9-8516-13298c687a3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:01.107212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f9cbc3-8f95-4ac9-8516-13298c687a3f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:01.115351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 114/228] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:01.120903 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9cbc3-8f95-4ac9-8516-13298c687a3f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:01.144848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afb166-a863-4dbf-9944-315ca7f777ba None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:01.187103 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:01.230331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f9cbc3-8f95-4ac9-8516-13298c687a3f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:01.252024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0570955-77c8-4f1a-9afc-cfb6a5370a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:01.252926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0570955-77c8-4f1a-9afc-cfb6a5370a58 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:01.253275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0570955-77c8-4f1a-9afc-cfb6a5370a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:01.253623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0570955-77c8-4f1a-9afc-cfb6a5370a58 None admin] PATH_INFO: `/v3/users/683477f356844d4e80bb88115e1d8de4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:01.254869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0570955-77c8-4f1a-9afc-cfb6a5370a58 None admin] RBAC: Authorizing `identity:get_user(user_id=683477f356844d4e80bb88115e1d8de4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:01.256322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0570955-77c8-4f1a-9afc-cfb6a5370a58 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:01.257993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 115/229] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:00 2022] GET /identity/v3/users/683477f356844d4e80bb88115e1d8de4 => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:01.263676 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0570955-77c8-4f1a-9afc-cfb6a5370a58 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:01.266791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a1d8-8f43-4bb9-b0cc-774ab61972af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:01.267771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a1d8-8f43-4bb9-b0cc-774ab61972af None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:01.268127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a1d8-8f43-4bb9-b0cc-774ab61972af None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:01.268474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a1d8-8f43-4bb9-b0cc-774ab61972af None admin] PATH_INFO: `/v3/users/683477f356844d4e80bb88115e1d8de4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:01.270337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23a1d8-8f43-4bb9-b0cc-774ab61972af None admin] RBAC: Authorizing `identity:get_user(user_id=683477f356844d4e80bb88115e1d8de4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:01.272096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23a1d8-8f43-4bb9-b0cc-774ab61972af None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:01.274480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 115/230] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:01 2022] GET /identity/v3/users/683477f356844d4e80bb88115e1d8de4 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:01.280985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a1d8-8f43-4bb9-b0cc-774ab61972af None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:01.421912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a1d8-8f43-4bb9-b0cc-774ab61972af None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:01.451662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:01.461206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272db614-0af7-47dc-9a47-79d57ce273ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:01.462370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272db614-0af7-47dc-9a47-79d57ce273ce None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:01.462808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272db614-0af7-47dc-9a47-79d57ce273ce None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:01.463566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272db614-0af7-47dc-9a47-79d57ce273ce None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:01.464989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272db614-0af7-47dc-9a47-79d57ce273ce None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:01.466591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272db614-0af7-47dc-9a47-79d57ce273ce None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:01.468844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 116/231] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:01 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:01.475352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272db614-0af7-47dc-9a47-79d57ce273ce None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:01.514467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0570955-77c8-4f1a-9afc-cfb6a5370a58 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:01.546891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:01.557177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a94e0-ca90-4d13-b9ba-efb8517dbac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:01.562254 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a94e0-ca90-4d13-b9ba-efb8517dbac2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:01.562542 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a94e0-ca90-4d13-b9ba-efb8517dbac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:01.562855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a94e0-ca90-4d13-b9ba-efb8517dbac2 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:01.564135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a94e0-ca90-4d13-b9ba-efb8517dbac2 None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:01.565597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a94e0-ca90-4d13-b9ba-efb8517dbac2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:01.568427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 116/232] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:01 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:01.574803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a94e0-ca90-4d13-b9ba-efb8517dbac2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:01.692358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a94e0-ca90-4d13-b9ba-efb8517dbac2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:01.716844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272db614-0af7-47dc-9a47-79d57ce273ce None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:01.732113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbeab24-2e7a-43fd-a7ce-360e2be7a7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:01.733062 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeab24-2e7a-43fd-a7ce-360e2be7a7d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:01.733343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeab24-2e7a-43fd-a7ce-360e2be7a7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:01.733618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeab24-2e7a-43fd-a7ce-360e2be7a7d5 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c/users/683477f356844d4e80bb88115e1d8de4/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:01.735634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbeab24-2e7a-43fd-a7ce-360e2be7a7d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c862664ec560416a936d940af366502c, user_id=683477f356844d4e80bb88115e1d8de4, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:01.737091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbeab24-2e7a-43fd-a7ce-360e2be7a7d5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:01.766360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 117/233] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:22:01 2022] PUT /identity/v3/projects/c862664ec560416a936d940af366502c/users/683477f356844d4e80bb88115e1d8de4/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:01.777984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2f042-d9cf-4385-8f25-bbaaf1656ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:01.779119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2f042-d9cf-4385-8f25-bbaaf1656ecd None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:01.779448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2f042-d9cf-4385-8f25-bbaaf1656ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:01.779784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2f042-d9cf-4385-8f25-bbaaf1656ecd None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c/users/683477f356844d4e80bb88115e1d8de4/roles/3531cd7306964cc5967f92c559890ae5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:01.782380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2f042-d9cf-4385-8f25-bbaaf1656ecd None admin] RBAC: Authorizing `identity:create_grant(project_id=c862664ec560416a936d940af366502c, user_id=683477f356844d4e80bb88115e1d8de4, role_id=3531cd7306964cc5967f92c559890ae5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:01.784159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2f042-d9cf-4385-8f25-bbaaf1656ecd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:01.823734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 117/234] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:22:01 2022] PUT /identity/v3/projects/c862664ec560416a936d940af366502c/users/683477f356844d4e80bb88115e1d8de4/roles/3531cd7306964cc5967f92c559890ae5 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:02.406383 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698b9b7-daee-4d4e-a2e7-37a37c026fe6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:02.406811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698b9b7-daee-4d4e-a2e7-37a37c026fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:02.406998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698b9b7-daee-4d4e-a2e7-37a37c026fe6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:02.408355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 118/235] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:02.413843 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dd5f3-2412-482f-aa7d-6da955542fe3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:02.414290 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dd5f3-2412-482f-aa7d-6da955542fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:02.414384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dd5f3-2412-482f-aa7d-6da955542fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:02.435712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-625dd5f3-2412-482f-aa7d-6da955542fe3 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:02.436925 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-625dd5f3-2412-482f-aa7d-6da955542fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:02.472461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a758f-bb28-47e9-97b4-96a5237c60cc None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:02.472751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a758f-bb28-47e9-97b4-96a5237c60cc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:02.473012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713a758f-bb28-47e9-97b4-96a5237c60cc None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:02.474403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 119/236] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:22:02.479744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e583582-40a5-4697-a287-b86856e25e20 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:02.480131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e583582-40a5-4697-a287-b86856e25e20 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:02.480413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e583582-40a5-4697-a287-b86856e25e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:02.499766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:02.512133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7e583582-40a5-4697-a287-b86856e25e20 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:02.513620 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7e583582-40a5-4697-a287-b86856e25e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:02.527890 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 118/237] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:02.571617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:02.573124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dd5f3-2412-482f-aa7d-6da955542fe3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:02.584308 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 120/238] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:02.624173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e583582-40a5-4697-a287-b86856e25e20 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:02.686804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dd5f3-2412-482f-aa7d-6da955542fe3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:02.720437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629102f-f028-4211-8d38-6246f78e3ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:02.721240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629102f-f028-4211-8d38-6246f78e3ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:02.721524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629102f-f028-4211-8d38-6246f78e3ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:02.721791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629102f-f028-4211-8d38-6246f78e3ba8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:02.723199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629102f-f028-4211-8d38-6246f78e3ba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:02.724864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629102f-f028-4211-8d38-6246f78e3ba8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:02.726514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 119/239] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:02.731940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629102f-f028-4211-8d38-6246f78e3ba8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:02.792739 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:02.841894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e583582-40a5-4697-a287-b86856e25e20 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:02.845508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629102f-f028-4211-8d38-6246f78e3ba8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:02.883339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3907245-82df-49fb-ad3a-49a77500116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:02.884110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3907245-82df-49fb-ad3a-49a77500116a None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:02.884394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3907245-82df-49fb-ad3a-49a77500116a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:02.884669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3907245-82df-49fb-ad3a-49a77500116a None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:02.885390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3907245-82df-49fb-ad3a-49a77500116a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:02.887080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3907245-82df-49fb-ad3a-49a77500116a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:02.904331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2ffc2-96c2-4a5d-ac47-691c455d3e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:02.905156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2ffc2-96c2-4a5d-ac47-691c455d3e92 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:02.905445 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2ffc2-96c2-4a5d-ac47-691c455d3e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:02.905734 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2ffc2-96c2-4a5d-ac47-691c455d3e92 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:02.909764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 120/240] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:22:02 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:22:02.929108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2ffc2-96c2-4a5d-ac47-691c455d3e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-aeb2ffc2-96c2-4a5d-ac47-691c455d3e92 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:02.945494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:02.946439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:02.948082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 121/241] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:02.953076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3907245-82df-49fb-ad3a-49a77500116a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:03.147089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3907245-82df-49fb-ad3a-49a77500116a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:03.164043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:03.191212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392098f9-d1a6-4481-b6c5-ad601f1546bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:03.191894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392098f9-d1a6-4481-b6c5-ad601f1546bd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:03.192179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392098f9-d1a6-4481-b6c5-ad601f1546bd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:03.192455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392098f9-d1a6-4481-b6c5-ad601f1546bd None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:03.193216 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392098f9-d1a6-4481-b6c5-ad601f1546bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:03.193527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392098f9-d1a6-4481-b6c5-ad601f1546bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:03.197571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392098f9-d1a6-4481-b6c5-ad601f1546bd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:03.207189 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 121/242] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:03.212805 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2ffc2-96c2-4a5d-ac47-691c455d3e92 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:03.260281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5eb76-687d-464c-8232-48f284e8e3b1 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:03.260951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5eb76-687d-464c-8232-48f284e8e3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:03.261440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5eb76-687d-464c-8232-48f284e8e3b1 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:03.263355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 122/243] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:22:03.269865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df43b00-5300-4115-8dca-a4618d479828 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:03.270238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df43b00-5300-4115-8dca-a4618d479828 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:03.271618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df43b00-5300-4115-8dca-a4618d479828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:03.293220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:03.299650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7df43b00-5300-4115-8dca-a4618d479828 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:03.301211 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7df43b00-5300-4115-8dca-a4618d479828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:03.364272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2ffc2-96c2-4a5d-ac47-691c455d3e92 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:03.377334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 123/244] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:03.400406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb06fd-ce89-4e9d-bcd8-aeefd020b6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:03.402088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df43b00-5300-4115-8dca-a4618d479828 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:03.403176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb06fd-ce89-4e9d-bcd8-aeefd020b6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:03.403460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb06fd-ce89-4e9d-bcd8-aeefd020b6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:03.403733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb06fd-ce89-4e9d-bcd8-aeefd020b6a4 None admin] PATH_INFO: `/v3/users/ee8c39cb257f45ad835ad20d7e005ae1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:03.418034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb06fd-ce89-4e9d-bcd8-aeefd020b6a4 None admin] RBAC: Authorizing `identity:get_user(user_id=ee8c39cb257f45ad835ad20d7e005ae1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:03.419577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb06fd-ce89-4e9d-bcd8-aeefd020b6a4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:03.421319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 122/245] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:03 2022] GET /identity/v3/users/ee8c39cb257f45ad835ad20d7e005ae1 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:03.427067 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb06fd-ce89-4e9d-bcd8-aeefd020b6a4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:03.540937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df43b00-5300-4115-8dca-a4618d479828 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:03.542324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:03.550279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb06fd-ce89-4e9d-bcd8-aeefd020b6a4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:03.560299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:03.579677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a816e-b849-42ef-b542-6ca8fe2b3beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:03.581354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a816e-b849-42ef-b542-6ca8fe2b3beb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:03.581784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a816e-b849-42ef-b542-6ca8fe2b3beb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:03.582193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a816e-b849-42ef-b542-6ca8fe2b3beb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:03.587896 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9526f3d-ab2a-4a30-9c0b-d61479204279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:03.588875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9526f3d-ab2a-4a30-9c0b-d61479204279 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:03.589260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9526f3d-ab2a-4a30-9c0b-d61479204279 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:03.589572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9526f3d-ab2a-4a30-9c0b-d61479204279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:03.595127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a816e-b849-42ef-b542-6ca8fe2b3beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3c1a816e-b849-42ef-b542-6ca8fe2b3beb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:03.603200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:03.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:03.604936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:03.607392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 124/246] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:03.613299 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a816e-b849-42ef-b542-6ca8fe2b3beb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:03.666085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9526f3d-ab2a-4a30-9c0b-d61479204279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:03.667629 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9526f3d-ab2a-4a30-9c0b-d61479204279 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:03.728572 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a816e-b849-42ef-b542-6ca8fe2b3beb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:03.778717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 123/247] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 08:22:03.782972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613285b-a503-48f4-95f8-4b6306499c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:03.783758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613285b-a503-48f4-95f8-4b6306499c07 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:03.784136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613285b-a503-48f4-95f8-4b6306499c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:03.784533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613285b-a503-48f4-95f8-4b6306499c07 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:03.785387 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613285b-a503-48f4-95f8-4b6306499c07 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:03.785789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613285b-a503-48f4-95f8-4b6306499c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:03.787310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3613285b-a503-48f4-95f8-4b6306499c07 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:03.790523 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9526f3d-ab2a-4a30-9c0b-d61479204279 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:03.797184 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 125/248] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:03.803084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613285b-a503-48f4-95f8-4b6306499c07 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:03.813537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:03.852181 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:03.969662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9526f3d-ab2a-4a30-9c0b-d61479204279 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:04.023936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f810a8-ddd2-4b38-92b0-3b875fd3393d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:04.024741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f810a8-ddd2-4b38-92b0-3b875fd3393d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:04.025021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f810a8-ddd2-4b38-92b0-3b875fd3393d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:04.025290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f810a8-ddd2-4b38-92b0-3b875fd3393d None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:04.031421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f810a8-ddd2-4b38-92b0-3b875fd3393d None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:04.032892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f810a8-ddd2-4b38-92b0-3b875fd3393d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:04.035041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 124/249] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:03 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:04.041430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f810a8-ddd2-4b38-92b0-3b875fd3393d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:04.079773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613285b-a503-48f4-95f8-4b6306499c07 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:04.122034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a969d-5c24-4e71-8159-d74c69b056d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:04.122944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a969d-5c24-4e71-8159-d74c69b056d0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:04.123267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a969d-5c24-4e71-8159-d74c69b056d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:04.123650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a969d-5c24-4e71-8159-d74c69b056d0 None admin] PATH_INFO: `/v3/users/584a201ce6e74faab6f445189615b23d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:04.124920 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a969d-5c24-4e71-8159-d74c69b056d0 None admin] RBAC: Authorizing `identity:get_user(user_id=584a201ce6e74faab6f445189615b23d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:04.126470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a969d-5c24-4e71-8159-d74c69b056d0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:04.128543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 126/250] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:03 2022] GET /identity/v3/users/584a201ce6e74faab6f445189615b23d => generated 297 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:04.135113 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a969d-5c24-4e71-8159-d74c69b056d0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:04.180492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:04.225750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f810a8-ddd2-4b38-92b0-3b875fd3393d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:04.290243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f34fa-e22d-4de9-8bf6-6fc58fc6f943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:04.290841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f34fa-e22d-4de9-8bf6-6fc58fc6f943 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:04.291122 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f34fa-e22d-4de9-8bf6-6fc58fc6f943 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:04.291392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f34fa-e22d-4de9-8bf6-6fc58fc6f943 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:04.292241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f34fa-e22d-4de9-8bf6-6fc58fc6f943 None admin] RBAC: Adding query filter params (role.id=9a50f15db31f490294f370186ce2d7ff, scope.project.id=c862664ec560416a936d940af366502c, user.id=ee8c39cb257f45ad835ad20d7e005ae1) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:04.292550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f34fa-e22d-4de9-8bf6-6fc58fc6f943 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:04.294012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f34fa-e22d-4de9-8bf6-6fc58fc6f943 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:04.302675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:04.304623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 125/251] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:04 2022] GET /identity/v3/role_assignments?user.id=ee8c39cb257f45ad835ad20d7e005ae1&scope.project.id=c862664ec560416a936d940af366502c&role.id=9a50f15db31f490294f370186ce2d7ff => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:04.316576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a969d-5c24-4e71-8159-d74c69b056d0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:04.377035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8b015-f26b-4f55-8248-bf72f87629a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:04.377919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8b015-f26b-4f55-8248-bf72f87629a2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:04.378315 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8b015-f26b-4f55-8248-bf72f87629a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:04.378581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8b015-f26b-4f55-8248-bf72f87629a2 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:04.383452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8b015-f26b-4f55-8248-bf72f87629a2 None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:04.384942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8b015-f26b-4f55-8248-bf72f87629a2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:04.386829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 127/252] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:04 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:04.392901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f34fa-e22d-4de9-8bf6-6fc58fc6f943 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:04.536727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f34fa-e22d-4de9-8bf6-6fc58fc6f943 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:04.580091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fceb72b-af19-4b9b-a190-3b6bdd4a36b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:04.580925 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceb72b-af19-4b9b-a190-3b6bdd4a36b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:04.581327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceb72b-af19-4b9b-a190-3b6bdd4a36b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:04.581561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fceb72b-af19-4b9b-a190-3b6bdd4a36b3 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c/users/584a201ce6e74faab6f445189615b23d/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:04.583893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fceb72b-af19-4b9b-a190-3b6bdd4a36b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c862664ec560416a936d940af366502c, user_id=584a201ce6e74faab6f445189615b23d, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:04.585374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fceb72b-af19-4b9b-a190-3b6bdd4a36b3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:04.611899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:04.617299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 126/253] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:22:04 2022] PUT /identity/v3/projects/c862664ec560416a936d940af366502c/users/584a201ce6e74faab6f445189615b23d/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:04.625960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee727515-5411-4641-9c88-c40daf5e5a72 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:04.626530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee727515-5411-4641-9c88-c40daf5e5a72 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:04.626605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee727515-5411-4641-9c88-c40daf5e5a72 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:04.627996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 128/254] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:04.634230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4d35f-7f6c-46ab-8a1e-a3c65ff3ceae None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:04.634553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4d35f-7f6c-46ab-8a1e-a3c65ff3ceae None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:04.634831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4d35f-7f6c-46ab-8a1e-a3c65ff3ceae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:04.656717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-42e4d35f-7f6c-46ab-8a1e-a3c65ff3ceae None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:04.658006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-42e4d35f-7f6c-46ab-8a1e-a3c65ff3ceae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:04.686819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b4cf8-dd7d-4d81-8d2c-5a9416fbb084 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:04.687176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b4cf8-dd7d-4d81-8d2c-5a9416fbb084 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:04.687381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b4cf8-dd7d-4d81-8d2c-5a9416fbb084 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:04.688781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 129/255] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:22:04.695920 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb786c-ad55-4b05-8adf-21812c8a6c07 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:04.696211 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb786c-ad55-4b05-8adf-21812c8a6c07 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:04.696465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb786c-ad55-4b05-8adf-21812c8a6c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:04.722673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2aeb786c-ad55-4b05-8adf-21812c8a6c07 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:04.726505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2aeb786c-ad55-4b05-8adf-21812c8a6c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:04.777534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 127/256] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:04.821263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4d35f-7f6c-46ab-8a1e-a3c65ff3ceae None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:04.873194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 130/257] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:04.926866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb786c-ad55-4b05-8adf-21812c8a6c07 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:04.983871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4d35f-7f6c-46ab-8a1e-a3c65ff3ceae None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:05.028882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:05.038428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43917344-9ee5-4fb8-b66a-929a197b6ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:05.039371 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43917344-9ee5-4fb8-b66a-929a197b6ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:05.039659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43917344-9ee5-4fb8-b66a-929a197b6ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:05.039937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43917344-9ee5-4fb8-b66a-929a197b6ca6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:05.051088 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43917344-9ee5-4fb8-b66a-929a197b6ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-43917344-9ee5-4fb8-b66a-929a197b6ca6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:05.060466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:05.061365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:05.062156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 128/258] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:22:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:05.068467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43917344-9ee5-4fb8-b66a-929a197b6ca6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:05.115956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:05.132712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb786c-ad55-4b05-8adf-21812c8a6c07 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:05.186554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f5f30-07f4-4d87-bfca-20fff9ee2fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:05.187407 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f5f30-07f4-4d87-bfca-20fff9ee2fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:05.187689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f5f30-07f4-4d87-bfca-20fff9ee2fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:05.187960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f5f30-07f4-4d87-bfca-20fff9ee2fb0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:05.202589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f5f30-07f4-4d87-bfca-20fff9ee2fb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e16f5f30-07f4-4d87-bfca-20fff9ee2fb0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:05.215392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:05.217073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:05.219519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 131/259] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:22:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:05.226810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f5f30-07f4-4d87-bfca-20fff9ee2fb0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:05.278508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43917344-9ee5-4fb8-b66a-929a197b6ca6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:05.341702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72fcca-5313-40c0-9ecb-e9b9fb63af6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:05.342955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72fcca-5313-40c0-9ecb-e9b9fb63af6b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:05.344212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72fcca-5313-40c0-9ecb-e9b9fb63af6b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:05.344688 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72fcca-5313-40c0-9ecb-e9b9fb63af6b None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:05.349811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72fcca-5313-40c0-9ecb-e9b9fb63af6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:05.350346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72fcca-5313-40c0-9ecb-e9b9fb63af6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:05.352285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72fcca-5313-40c0-9ecb-e9b9fb63af6b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:05.366486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 129/260] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:05.376093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72fcca-5313-40c0-9ecb-e9b9fb63af6b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:05.412769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f5f30-07f4-4d87-bfca-20fff9ee2fb0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:05.470410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:05.499754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd3e74-e134-4fe5-8a0b-b819f08fb704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:05.500792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd3e74-e134-4fe5-8a0b-b819f08fb704 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:05.501272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd3e74-e134-4fe5-8a0b-b819f08fb704 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:05.501742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd3e74-e134-4fe5-8a0b-b819f08fb704 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:05.502774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd3e74-e134-4fe5-8a0b-b819f08fb704 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:05.503863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd3e74-e134-4fe5-8a0b-b819f08fb704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:05.506221 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd3e74-e134-4fe5-8a0b-b819f08fb704 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:05.529096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 132/261] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:05.540567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd3e74-e134-4fe5-8a0b-b819f08fb704 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:05.584709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:05.613519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72fcca-5313-40c0-9ecb-e9b9fb63af6b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:05.703505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a538a89-88b7-43bf-a2f2-f88aec7ac1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:05.704763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a538a89-88b7-43bf-a2f2-f88aec7ac1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:05.705267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a538a89-88b7-43bf-a2f2-f88aec7ac1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:05.705737 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a538a89-88b7-43bf-a2f2-f88aec7ac1b4 None admin] PATH_INFO: `/v3/users/683477f356844d4e80bb88115e1d8de4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:05.707903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a538a89-88b7-43bf-a2f2-f88aec7ac1b4 None admin] RBAC: Authorizing `identity:get_user(user_id=683477f356844d4e80bb88115e1d8de4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:05.709578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a538a89-88b7-43bf-a2f2-f88aec7ac1b4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:05.711499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 130/262] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:05 2022] GET /identity/v3/users/683477f356844d4e80bb88115e1d8de4 => generated 283 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:05.717241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a538a89-88b7-43bf-a2f2-f88aec7ac1b4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:05.723550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd3e74-e134-4fe5-8a0b-b819f08fb704 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:05.801774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ddeb9-e886-40e1-b5d9-a1d94eb91dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:05.802837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ddeb9-e886-40e1-b5d9-a1d94eb91dbb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:05.803130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ddeb9-e886-40e1-b5d9-a1d94eb91dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:05.803403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ddeb9-e886-40e1-b5d9-a1d94eb91dbb None admin] PATH_INFO: `/v3/users/683477f356844d4e80bb88115e1d8de4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:05.805927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ddeb9-e886-40e1-b5d9-a1d94eb91dbb None admin] RBAC: Authorizing `identity:get_user(user_id=683477f356844d4e80bb88115e1d8de4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:05.808304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ddeb9-e886-40e1-b5d9-a1d94eb91dbb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:05.810293 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 133/263] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:05 2022] GET /identity/v3/users/683477f356844d4e80bb88115e1d8de4 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:05.815287 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ddeb9-e886-40e1-b5d9-a1d94eb91dbb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:05.908253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a538a89-88b7-43bf-a2f2-f88aec7ac1b4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:05.950722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:05.976905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c78dd-dea6-4830-9eca-0d474d100624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:05.977931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c78dd-dea6-4830-9eca-0d474d100624 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:05.978409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c78dd-dea6-4830-9eca-0d474d100624 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:05.978900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c78dd-dea6-4830-9eca-0d474d100624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:05.985570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:06.020054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ddeb9-e886-40e1-b5d9-a1d94eb91dbb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:06.078536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c78dd-dea6-4830-9eca-0d474d100624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:06.080406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c78dd-dea6-4830-9eca-0d474d100624 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:06.090719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bb5ef-d88d-43b9-970f-636607f0ea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:06.090719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bb5ef-d88d-43b9-970f-636607f0ea1a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:06.090719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bb5ef-d88d-43b9-970f-636607f0ea1a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:06.090719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bb5ef-d88d-43b9-970f-636607f0ea1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:06.180742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987bb5ef-d88d-43b9-970f-636607f0ea1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:06.182230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987bb5ef-d88d-43b9-970f-636607f0ea1a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:06.236744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 131/264] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:06.239079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c0fc7-a1e5-4a54-8a10-b369ebdc4e75 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:06.239574 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c0fc7-a1e5-4a54-8a10-b369ebdc4e75 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:06.240048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c0fc7-a1e5-4a54-8a10-b369ebdc4e75 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:06.241832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 132/265] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:06.243228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c0fc7-a1e5-4a54-8a10-b369ebdc4e75 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:06.267035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:06.321543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:06.363137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 134/266] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 08:22:06.364779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b504d96-b0a7-492f-a73f-d26aa08a944a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:06.367662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b504d96-b0a7-492f-a73f-d26aa08a944a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:06.368140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b504d96-b0a7-492f-a73f-d26aa08a944a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:06.384698 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c0fc7-a1e5-4a54-8a10-b369ebdc4e75 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:06.403706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0b504d96-b0a7-492f-a73f-d26aa08a944a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:06.404926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0b504d96-b0a7-492f-a73f-d26aa08a944a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:06.420456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce9a7c-ce0f-4ad1-a981-fe2c7f234803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:06.421765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce9a7c-ce0f-4ad1-a981-fe2c7f234803 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:06.422260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce9a7c-ce0f-4ad1-a981-fe2c7f234803 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:06.422744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce9a7c-ce0f-4ad1-a981-fe2c7f234803 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:06.424224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce9a7c-ce0f-4ad1-a981-fe2c7f234803 None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:06.426009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce9a7c-ce0f-4ad1-a981-fe2c7f234803 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:06.428275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 133/267] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:06 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:06.429287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce9a7c-ce0f-4ad1-a981-fe2c7f234803 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:06.524752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 135/268] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:06.525540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b504d96-b0a7-492f-a73f-d26aa08a944a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:06.541408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce9a7c-ce0f-4ad1-a981-fe2c7f234803 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:06.544803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:06.586590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f42a5-7f49-41c1-bda9-06bd8260044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:06.587964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f42a5-7f49-41c1-bda9-06bd8260044e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:06.589091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f42a5-7f49-41c1-bda9-06bd8260044e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:06.589504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f42a5-7f49-41c1-bda9-06bd8260044e None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:06.591081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f42a5-7f49-41c1-bda9-06bd8260044e None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:06.592954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f42a5-7f49-41c1-bda9-06bd8260044e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:06.596047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 134/269] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:06 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:06.598674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b5dcb-8043-465b-a48f-e998aae8b8f5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:06.599138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b5dcb-8043-465b-a48f-e998aae8b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:06.599565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b5dcb-8043-465b-a48f-e998aae8b8f5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:06.601136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 135/270] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:06.601995 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b5dcb-8043-465b-a48f-e998aae8b8f5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:06.635989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b504d96-b0a7-492f-a73f-d26aa08a944a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:06.647346 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:06.679737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899d74e-58bb-40c8-8994-df39b354a3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:06.680586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899d74e-58bb-40c8-8994-df39b354a3ef None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:06.680966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899d74e-58bb-40c8-8994-df39b354a3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:06.681375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899d74e-58bb-40c8-8994-df39b354a3ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:06.682428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899d74e-58bb-40c8-8994-df39b354a3ef None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=c862664ec560416a936d940af366502c, user.id=683477f356844d4e80bb88115e1d8de4) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:06.683268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899d74e-58bb-40c8-8994-df39b354a3ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:06.685012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899d74e-58bb-40c8-8994-df39b354a3ef None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:06.697549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 136/271] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:06 2022] GET /identity/v3/role_assignments?user.id=683477f356844d4e80bb88115e1d8de4&scope.project.id=c862664ec560416a936d940af366502c&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:06.699585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899d74e-58bb-40c8-8994-df39b354a3ef None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:06.748545 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b5dcb-8043-465b-a48f-e998aae8b8f5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:06.820754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3013a-8b12-444e-b580-c367279cdb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:06.821493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3013a-8b12-444e-b580-c367279cdb7f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:06.821775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3013a-8b12-444e-b580-c367279cdb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:06.822047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3013a-8b12-444e-b580-c367279cdb7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:06.837303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899d74e-58bb-40c8-8994-df39b354a3ef None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:06.844394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3013a-8b12-444e-b580-c367279cdb7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-8db3013a-8b12-444e-b580-c367279cdb7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:06.860721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:06.861664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:06.862465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:06.871211 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 136/272] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:06.872457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576afd28-b09e-479b-91b1-9fb015668ad8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:06.872750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576afd28-b09e-479b-91b1-9fb015668ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:06.873021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576afd28-b09e-479b-91b1-9fb015668ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:06.877515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4541b6-e7d7-4620-8c5d-7c6e802ab0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:06.879160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4541b6-e7d7-4620-8c5d-7c6e802ab0ec None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:06.886344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4541b6-e7d7-4620-8c5d-7c6e802ab0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:06.886344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4541b6-e7d7-4620-8c5d-7c6e802ab0ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:06.887742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4541b6-e7d7-4620-8c5d-7c6e802ab0ec None admin] RBAC: Adding query filter params (role.id=3531cd7306964cc5967f92c559890ae5, scope.project.id=c862664ec560416a936d940af366502c, user.id=683477f356844d4e80bb88115e1d8de4) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:06.888069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4541b6-e7d7-4620-8c5d-7c6e802ab0ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:06.889562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4541b6-e7d7-4620-8c5d-7c6e802ab0ec None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:06.895827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-576afd28-b09e-479b-91b1-9fb015668ad8 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:06.897287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-576afd28-b09e-479b-91b1-9fb015668ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:06.904181 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 137/273] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:06 2022] GET /identity/v3/role_assignments?user.id=683477f356844d4e80bb88115e1d8de4&scope.project.id=c862664ec560416a936d940af366502c&role.id=3531cd7306964cc5967f92c559890ae5 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:06.905046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4541b6-e7d7-4620-8c5d-7c6e802ab0ec None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:06.911096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:07.011082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:07.064186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 137/274] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:07.091492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4541b6-e7d7-4620-8c5d-7c6e802ab0ec None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:07.111842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576afd28-b09e-479b-91b1-9fb015668ad8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:07.142182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e634f6-4032-44e0-822b-4d1c3405e623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:07.142902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e634f6-4032-44e0-822b-4d1c3405e623 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.143187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e634f6-4032-44e0-822b-4d1c3405e623 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.143464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e634f6-4032-44e0-822b-4d1c3405e623 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.144555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e634f6-4032-44e0-822b-4d1c3405e623 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:07.144555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e634f6-4032-44e0-822b-4d1c3405e623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:07.145969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e634f6-4032-44e0-822b-4d1c3405e623 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:07.155319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 138/275] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:07.164332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e634f6-4032-44e0-822b-4d1c3405e623 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:07.265944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576afd28-b09e-479b-91b1-9fb015668ad8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:07.279841 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:07.283455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e634f6-4032-44e0-822b-4d1c3405e623 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:07.306315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60243769-9d5a-4a18-9dbb-4995ac404b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:07.307174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60243769-9d5a-4a18-9dbb-4995ac404b1c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.307457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60243769-9d5a-4a18-9dbb-4995ac404b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.307731 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60243769-9d5a-4a18-9dbb-4995ac404b1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.308897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60243769-9d5a-4a18-9dbb-4995ac404b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:07.310351 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60243769-9d5a-4a18-9dbb-4995ac404b1c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:07.312068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 138/276] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:07.313167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d8809-faed-45d2-82a8-e002b195982d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.313446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d8809-faed-45d2-82a8-e002b195982d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.313718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d8809-faed-45d2-82a8-e002b195982d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.319172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 139/277] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:07.320648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d8809-faed-45d2-82a8-e002b195982d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:07.326037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a44899-0931-4f91-977f-e3fc849e3bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:07.326960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a44899-0931-4f91-977f-e3fc849e3bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.327244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a44899-0931-4f91-977f-e3fc849e3bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.327520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a44899-0931-4f91-977f-e3fc849e3bb7 None admin] PATH_INFO: `/v3/users/b9530ded36ca4ae18cc3e7b07a3b7719` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.338924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:07.341914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a44899-0931-4f91-977f-e3fc849e3bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=b9530ded36ca4ae18cc3e7b07a3b7719)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:07.343404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a44899-0931-4f91-977f-e3fc849e3bb7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:07.345122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 139/278] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:07 2022] GET /identity/v3/users/b9530ded36ca4ae18cc3e7b07a3b7719 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:07.346298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5c4e3-b3fa-41b9-b9c9-c1b0a9331088 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.346583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5c4e3-b3fa-41b9-b9c9-c1b0a9331088 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.347105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5c4e3-b3fa-41b9-b9c9-c1b0a9331088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.363263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cab5c4e3-b3fa-41b9-b9c9-c1b0a9331088 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:07.364444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cab5c4e3-b3fa-41b9-b9c9-c1b0a9331088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:07.436252 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 140/279] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:07.436444 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d8809-faed-45d2-82a8-e002b195982d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:07.439578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5c4e3-b3fa-41b9-b9c9-c1b0a9331088 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:07.475988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138b377-4378-455d-8c87-deb21a008934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:07.476671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138b377-4378-455d-8c87-deb21a008934 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.476953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138b377-4378-455d-8c87-deb21a008934 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.477230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138b377-4378-455d-8c87-deb21a008934 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.478238 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138b377-4378-455d-8c87-deb21a008934 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:07.479720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138b377-4378-455d-8c87-deb21a008934 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:07.505410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 140/280] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:22:07 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:22:07.506175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138b377-4378-455d-8c87-deb21a008934 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:07.559254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:07.572927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5c4e3-b3fa-41b9-b9c9-c1b0a9331088 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:07.595316 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:07.614078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b528b1-27e2-4c35-b010-80d0247e642f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:07.614879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b528b1-27e2-4c35-b010-80d0247e642f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.615166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b528b1-27e2-4c35-b010-80d0247e642f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.615440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b528b1-27e2-4c35-b010-80d0247e642f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.616294 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b528b1-27e2-4c35-b010-80d0247e642f None admin] RBAC: Adding query filter params (role.id=b0d377da1a7d4260be1f7ae0715fda91, user.id=b9530ded36ca4ae18cc3e7b07a3b7719, scope.system=all) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:07.616604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b528b1-27e2-4c35-b010-80d0247e642f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:07.618088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b528b1-27e2-4c35-b010-80d0247e642f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:07.624316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 141/281] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Jun 29 08:22:07 2022] GET /identity/v3/role_assignments?user.id=b9530ded36ca4ae18cc3e7b07a3b7719&scope.system=all&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:07.626023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e25b85-ca35-4257-83d9-61a4d0957d41 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.626540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e25b85-ca35-4257-83d9-61a4d0957d41 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.627019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e25b85-ca35-4257-83d9-61a4d0957d41 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.628780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 142/282] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:07.635238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8cd75-166f-4504-bab9-ebe7ab07b9ef None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.635857 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8cd75-166f-4504-bab9-ebe7ab07b9ef None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.636846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8cd75-166f-4504-bab9-ebe7ab07b9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.643417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138b377-4378-455d-8c87-deb21a008934 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:07.661832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dce8cd75-166f-4504-bab9-ebe7ab07b9ef None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:07.663798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dce8cd75-166f-4504-bab9-ebe7ab07b9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:07.691998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32d337-4f32-4ef6-af59-1cb508841f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:07.693169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32d337-4f32-4ef6-af59-1cb508841f37 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.693622 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32d337-4f32-4ef6-af59-1cb508841f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.694068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32d337-4f32-4ef6-af59-1cb508841f37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.707881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32d337-4f32-4ef6-af59-1cb508841f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-bd32d337-4f32-4ef6-af59-1cb508841f37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:07.717320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:07.718280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:07.719137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:07.721481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 141/283] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:07.729391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32d337-4f32-4ef6-af59-1cb508841f37 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:07.760111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 143/284] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:07.784528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8cd75-166f-4504-bab9-ebe7ab07b9ef None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:07.868967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32d337-4f32-4ef6-af59-1cb508841f37 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:07.882935 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:07.908787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8cd75-166f-4504-bab9-ebe7ab07b9ef None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:07.912598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:07.922323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54991a3-1e36-41be-b465-9513b6070017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:07.923168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54991a3-1e36-41be-b465-9513b6070017 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.923449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54991a3-1e36-41be-b465-9513b6070017 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.923722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54991a3-1e36-41be-b465-9513b6070017 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.924480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54991a3-1e36-41be-b465-9513b6070017 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:07.924787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54991a3-1e36-41be-b465-9513b6070017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:07.926268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54991a3-1e36-41be-b465-9513b6070017 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:07.938313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 142/285] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:07.945671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54991a3-1e36-41be-b465-9513b6070017 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:07.958684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cedb87-20f3-46db-8d28-5cf0017d8b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:07.959644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cedb87-20f3-46db-8d28-5cf0017d8b77 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:07.959927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cedb87-20f3-46db-8d28-5cf0017d8b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:07.960201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cedb87-20f3-46db-8d28-5cf0017d8b77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:07.971506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cedb87-20f3-46db-8d28-5cf0017d8b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-74cedb87-20f3-46db-8d28-5cf0017d8b77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:07.979654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:07.980559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:07.981325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 144/286] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:07.988161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cedb87-20f3-46db-8d28-5cf0017d8b77 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:08.054969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54991a3-1e36-41be-b465-9513b6070017 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:08.091465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cedb87-20f3-46db-8d28-5cf0017d8b77 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:08.093484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcec08-6746-4c5d-bfc4-96e005696d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:08.095500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcec08-6746-4c5d-bfc4-96e005696d14 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:08.096005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcec08-6746-4c5d-bfc4-96e005696d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:08.096471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcec08-6746-4c5d-bfc4-96e005696d14 None admin] PATH_INFO: `/v3/users/ee8c39cb257f45ad835ad20d7e005ae1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:08.097971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcec08-6746-4c5d-bfc4-96e005696d14 None admin] RBAC: Authorizing `identity:get_user(user_id=ee8c39cb257f45ad835ad20d7e005ae1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:08.100239 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcec08-6746-4c5d-bfc4-96e005696d14 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:08.103281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 143/287] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:07 2022] GET /identity/v3/users/ee8c39cb257f45ad835ad20d7e005ae1 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:08.113863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcec08-6746-4c5d-bfc4-96e005696d14 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:08.155562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a9c93-b626-4d9c-ae9e-8992152bb134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:08.156465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9c93-b626-4d9c-ae9e-8992152bb134 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:08.156465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9c93-b626-4d9c-ae9e-8992152bb134 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:08.156713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a9c93-b626-4d9c-ae9e-8992152bb134 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:08.157477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a9c93-b626-4d9c-ae9e-8992152bb134 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:08.157792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a9c93-b626-4d9c-ae9e-8992152bb134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:08.159225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a9c93-b626-4d9c-ae9e-8992152bb134 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:08.188772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 145/288] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:08.195721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a9c93-b626-4d9c-ae9e-8992152bb134 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:08.199299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:08.207854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:08.260171 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcec08-6746-4c5d-bfc4-96e005696d14 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:08.327599 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a9c93-b626-4d9c-ae9e-8992152bb134 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:08.331917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c49b1-63ed-40a3-a254-bb243fd84710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:08.333082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c49b1-63ed-40a3-a254-bb243fd84710 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:08.333670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c49b1-63ed-40a3-a254-bb243fd84710 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:08.334828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c49b1-63ed-40a3-a254-bb243fd84710 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:08.336414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c49b1-63ed-40a3-a254-bb243fd84710 None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:08.342702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c49b1-63ed-40a3-a254-bb243fd84710 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:08.344682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 144/289] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:08 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:08.350607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c49b1-63ed-40a3-a254-bb243fd84710 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:08.405698 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb3a2c-d310-49ad-880a-fbe6e5ca49d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:08.408152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb3a2c-d310-49ad-880a-fbe6e5ca49d9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:08.409881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb3a2c-d310-49ad-880a-fbe6e5ca49d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:08.410433 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb3a2c-d310-49ad-880a-fbe6e5ca49d9 None admin] PATH_INFO: `/v3/users/584a201ce6e74faab6f445189615b23d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:08.412908 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb3a2c-d310-49ad-880a-fbe6e5ca49d9 None admin] RBAC: Authorizing `identity:get_user(user_id=584a201ce6e74faab6f445189615b23d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:08.415906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb3a2c-d310-49ad-880a-fbe6e5ca49d9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:08.422461 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 146/290] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:08 2022] GET /identity/v3/users/584a201ce6e74faab6f445189615b23d => generated 297 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:08.445031 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb3a2c-d310-49ad-880a-fbe6e5ca49d9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:08.528527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c49b1-63ed-40a3-a254-bb243fd84710 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:08.558906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:08.568913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76822cf-93a7-4baf-8584-fb18ecfce034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:08.570130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76822cf-93a7-4baf-8584-fb18ecfce034 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:08.570608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76822cf-93a7-4baf-8584-fb18ecfce034 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:08.571302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76822cf-93a7-4baf-8584-fb18ecfce034 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c/users/ee8c39cb257f45ad835ad20d7e005ae1/roles/9a50f15db31f490294f370186ce2d7ff` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:08.574867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76822cf-93a7-4baf-8584-fb18ecfce034 None admin] RBAC: Authorizing `identity:create_grant(project_id=c862664ec560416a936d940af366502c, user_id=ee8c39cb257f45ad835ad20d7e005ae1, role_id=9a50f15db31f490294f370186ce2d7ff)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:08.576636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76822cf-93a7-4baf-8584-fb18ecfce034 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:08.597543 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 145/291] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:22:08 2022] PUT /identity/v3/projects/c862664ec560416a936d940af366502c/users/ee8c39cb257f45ad835ad20d7e005ae1/roles/9a50f15db31f490294f370186ce2d7ff => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:22:08.600301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:08.646474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb3a2c-d310-49ad-880a-fbe6e5ca49d9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:08.696505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc145a-c4d1-4df3-80f2-ca3f1ffa41bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:08.697531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc145a-c4d1-4df3-80f2-ca3f1ffa41bb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:08.698039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc145a-c4d1-4df3-80f2-ca3f1ffa41bb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:08.698521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc145a-c4d1-4df3-80f2-ca3f1ffa41bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:08.778054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc145a-c4d1-4df3-80f2-ca3f1ffa41bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:08.779960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc145a-c4d1-4df3-80f2-ca3f1ffa41bb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:08.856584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:08.891002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 147/292] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:08.897649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76822cf-93a7-4baf-8584-fb18ecfce034 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:09.008443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76822cf-93a7-4baf-8584-fb18ecfce034 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:09.045307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d571c0-1d4d-4816-80d6-afe4a89b500e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:09.046533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d571c0-1d4d-4816-80d6-afe4a89b500e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:09.047052 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d571c0-1d4d-4816-80d6-afe4a89b500e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:09.047517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d571c0-1d4d-4816-80d6-afe4a89b500e None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:09.048921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d571c0-1d4d-4816-80d6-afe4a89b500e None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:09.051013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d571c0-1d4d-4816-80d6-afe4a89b500e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:09.053291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 146/293] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:08 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:09.059323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc145a-c4d1-4df3-80f2-ca3f1ffa41bb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:09.198254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc145a-c4d1-4df3-80f2-ca3f1ffa41bb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:09.244446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0ebe4-9d3a-4f3d-8e4c-1925c761072e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:09.245517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0ebe4-9d3a-4f3d-8e4c-1925c761072e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:09.245993 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0ebe4-9d3a-4f3d-8e4c-1925c761072e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:09.246459 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0ebe4-9d3a-4f3d-8e4c-1925c761072e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:09.247526 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0ebe4-9d3a-4f3d-8e4c-1925c761072e None admin] RBAC: Adding query filter params (role.id=b0d377da1a7d4260be1f7ae0715fda91, scope.project.id=c862664ec560416a936d940af366502c, user.id=584a201ce6e74faab6f445189615b23d) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:09.248049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0ebe4-9d3a-4f3d-8e4c-1925c761072e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:09.249694 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0ebe4-9d3a-4f3d-8e4c-1925c761072e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:09.256217 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 148/294] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:09 2022] GET /identity/v3/role_assignments?user.id=584a201ce6e74faab6f445189615b23d&scope.project.id=c862664ec560416a936d940af366502c&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:10.018466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6224-0cf9-43e0-a851-7da287a92a00 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.019079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6224-0cf9-43e0-a851-7da287a92a00 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.019187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df6224-0cf9-43e0-a851-7da287a92a00 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.020769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 147/295] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:10.026333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e95910-60db-485f-a22d-bfc702b9584d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.026777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e95910-60db-485f-a22d-bfc702b9584d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.027095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e95910-60db-485f-a22d-bfc702b9584d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.044136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-50e95910-60db-485f-a22d-bfc702b9584d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:10.045322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-50e95910-60db-485f-a22d-bfc702b9584d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:10.053367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cbe62-bc74-48ce-8ec2-e586138526fc None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.053652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cbe62-bc74-48ce-8ec2-e586138526fc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.053928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cbe62-bc74-48ce-8ec2-e586138526fc None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.055365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 148/296] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:22:10.060575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a1253-a30d-4f9b-8a41-ef2c3b0c6185 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.060860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a1253-a30d-4f9b-8a41-ef2c3b0c6185 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.061137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a1253-a30d-4f9b-8a41-ef2c3b0c6185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.078078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:10.078253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-770a1253-a30d-4f9b-8a41-ef2c3b0c6185 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:10.079822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-770a1253-a30d-4f9b-8a41-ef2c3b0c6185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:10.117257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 149/297] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:10.118738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58166c52-0288-402f-9abf-f9ef259c83fc None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.119031 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58166c52-0288-402f-9abf-f9ef259c83fc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.119304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58166c52-0288-402f-9abf-f9ef259c83fc None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.120698 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 150/298] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:10.125954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a3801-d6a7-4b89-80ed-4d682686efd5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.126193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a3801-d6a7-4b89-80ed-4d682686efd5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.126471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a3801-d6a7-4b89-80ed-4d682686efd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.138540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:10.144365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-296a3801-d6a7-4b89-80ed-4d682686efd5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:10.145577 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-296a3801-d6a7-4b89-80ed-4d682686efd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:10.164042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 149/299] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:10.165632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a1253-a30d-4f9b-8a41-ef2c3b0c6185 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:10.215158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 151/300] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:10.218498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a3801-d6a7-4b89-80ed-4d682686efd5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:10.282569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a1253-a30d-4f9b-8a41-ef2c3b0c6185 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:10.320681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d0042-0e29-4292-b40a-c8379e4d8b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:10.321575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d0042-0e29-4292-b40a-c8379e4d8b00 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.321858 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d0042-0e29-4292-b40a-c8379e4d8b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.322192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d0042-0e29-4292-b40a-c8379e4d8b00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.323302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d0042-0e29-4292-b40a-c8379e4d8b00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:10.324740 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d0042-0e29-4292-b40a-c8379e4d8b00 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:10.326428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 150/301] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:10.326922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:10.327173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d0042-0e29-4292-b40a-c8379e4d8b00 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:10.337179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a3801-d6a7-4b89-80ed-4d682686efd5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:10.374551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f00349-e271-4595-9615-4bf11a48e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:10.375384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f00349-e271-4595-9615-4bf11a48e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.375659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f00349-e271-4595-9615-4bf11a48e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.375933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f00349-e271-4595-9615-4bf11a48e0f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.386383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f00349-e271-4595-9615-4bf11a48e0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a3f00349-e271-4595-9615-4bf11a48e0f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:10.394256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:10.395177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 152/302] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:10.397314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f00349-e271-4595-9615-4bf11a48e0f2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:10.403010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:10.453500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d0042-0e29-4292-b40a-c8379e4d8b00 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:10.493677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be33489-a80d-40aa-8a03-440929f4447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:10.494486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be33489-a80d-40aa-8a03-440929f4447a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.494797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be33489-a80d-40aa-8a03-440929f4447a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.495086 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be33489-a80d-40aa-8a03-440929f4447a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.500542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f00349-e271-4595-9615-4bf11a48e0f2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:10.507138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be33489-a80d-40aa-8a03-440929f4447a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3be33489-a80d-40aa-8a03-440929f4447a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:10.515840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:10.516761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:10.517539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 151/303] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:10.518175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be33489-a80d-40aa-8a03-440929f4447a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:10.543454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bec08-648d-4b31-93c6-e2e0523284c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:10.544049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bec08-648d-4b31-93c6-e2e0523284c7 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.544336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bec08-648d-4b31-93c6-e2e0523284c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.544610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bec08-648d-4b31-93c6-e2e0523284c7 None admin] PATH_INFO: `/v3/groups` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.545309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bec08-648d-4b31-93c6-e2e0523284c7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:10.546803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bec08-648d-4b31-93c6-e2e0523284c7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:10.562245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 153/304] 10.222.0.27 () {60 vars in 1134 bytes} [Wed Jun 29 08:22:10 2022] POST /identity/v3/groups => generated 229 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:22:10.562245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bec08-648d-4b31-93c6-e2e0523284c7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:10.599425 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:10.650285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be33489-a80d-40aa-8a03-440929f4447a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:10.668457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:10.694884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a064f-cf27-4fbb-9c20-d8e5ce751a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:10.696032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a064f-cf27-4fbb-9c20-d8e5ce751a1d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.696032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a064f-cf27-4fbb-9c20-d8e5ce751a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.696451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a064f-cf27-4fbb-9c20-d8e5ce751a1d None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.697339 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a064f-cf27-4fbb-9c20-d8e5ce751a1d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:10.697739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a064f-cf27-4fbb-9c20-d8e5ce751a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:10.699332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a064f-cf27-4fbb-9c20-d8e5ce751a1d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:10.711579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 152/305] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:10.719984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a064f-cf27-4fbb-9c20-d8e5ce751a1d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:10.729796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bec08-648d-4b31-93c6-e2e0523284c7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:10.781352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1c8a-a406-48b1-abf9-0efd8a1a25f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:10.782041 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1c8a-a406-48b1-abf9-0efd8a1a25f6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.782329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1c8a-a406-48b1-abf9-0efd8a1a25f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.782627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af1c8a-a406-48b1-abf9-0efd8a1a25f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.783434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af1c8a-a406-48b1-abf9-0efd8a1a25f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:10.784677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af1c8a-a406-48b1-abf9-0efd8a1a25f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:10.786081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af1c8a-a406-48b1-abf9-0efd8a1a25f6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:10.795093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 154/306] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:10.800511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1c8a-a406-48b1-abf9-0efd8a1a25f6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:10.854128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a064f-cf27-4fbb-9c20-d8e5ce751a1d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:10.890968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cef88-69e2-4150-b110-b6bcfb1ed4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:10.891936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cef88-69e2-4150-b110-b6bcfb1ed4ae None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.892273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cef88-69e2-4150-b110-b6bcfb1ed4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.892555 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cef88-69e2-4150-b110-b6bcfb1ed4ae None admin] PATH_INFO: `/v3/users/875e76dd3cb741248f910c395193e006` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.896889 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:10.908682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cef88-69e2-4150-b110-b6bcfb1ed4ae None admin] RBAC: Authorizing `identity:get_user(user_id=875e76dd3cb741248f910c395193e006)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:10.910448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cef88-69e2-4150-b110-b6bcfb1ed4ae None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:10.912673 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 153/307] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:10 2022] GET /identity/v3/users/875e76dd3cb741248f910c395193e006 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:10.918349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cef88-69e2-4150-b110-b6bcfb1ed4ae None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:10.923078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af1c8a-a406-48b1-abf9-0efd8a1a25f6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:10.956917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:10.975028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bdce4-3f61-4bdc-8adb-2ea69b7a9c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:10.976883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bdce4-3f61-4bdc-8adb-2ea69b7a9c6a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:10.977390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bdce4-3f61-4bdc-8adb-2ea69b7a9c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:10.977858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bdce4-3f61-4bdc-8adb-2ea69b7a9c6a None admin] PATH_INFO: `/v3/users/b9530ded36ca4ae18cc3e7b07a3b7719` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:10.979934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bdce4-3f61-4bdc-8adb-2ea69b7a9c6a None admin] RBAC: Authorizing `identity:get_user(user_id=b9530ded36ca4ae18cc3e7b07a3b7719)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:10.981912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bdce4-3f61-4bdc-8adb-2ea69b7a9c6a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:10.985004 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 155/308] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:10 2022] GET /identity/v3/users/b9530ded36ca4ae18cc3e7b07a3b7719 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:10.990505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bdce4-3f61-4bdc-8adb-2ea69b7a9c6a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:11.047968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cef88-69e2-4150-b110-b6bcfb1ed4ae None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:11.091440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2e097-5566-446a-935b-b3755344b2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:11.092406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2e097-5566-446a-935b-b3755344b2aa None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:11.092692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2e097-5566-446a-935b-b3755344b2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:11.092966 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2e097-5566-446a-935b-b3755344b2aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:11.093816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2e097-5566-446a-935b-b3755344b2aa None admin] RBAC: Adding query filter params (role.id=9a50f15db31f490294f370186ce2d7ff, user.id=875e76dd3cb741248f910c395193e006, scope.system=all) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:11.094128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2e097-5566-446a-935b-b3755344b2aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:11.095759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2e097-5566-446a-935b-b3755344b2aa None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:11.102542 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 154/309] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Jun 29 08:22:10 2022] GET /identity/v3/role_assignments?user.id=875e76dd3cb741248f910c395193e006&scope.system=all&role.id=9a50f15db31f490294f370186ce2d7ff => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:11.110883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bdce4-3f61-4bdc-8adb-2ea69b7a9c6a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:11.116297 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d46ab-4a99-40b5-8980-3068c7e54d81 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:11.116619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d46ab-4a99-40b5-8980-3068c7e54d81 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:11.116853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d46ab-4a99-40b5-8980-3068c7e54d81 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:11.122650 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 155/310] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:22:11.124367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fef25-45c0-4fa9-b43b-6a14478866e8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:11.124482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fef25-45c0-4fa9-b43b-6a14478866e8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:11.124722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fef25-45c0-4fa9-b43b-6a14478866e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:11.142477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6a2fef25-45c0-4fa9-b43b-6a14478866e8 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:11.143801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6a2fef25-45c0-4fa9-b43b-6a14478866e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:11.154435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f32439-c987-46b0-8907-00c726179288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:11.156028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f32439-c987-46b0-8907-00c726179288 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:11.156515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f32439-c987-46b0-8907-00c726179288 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:11.156990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f32439-c987-46b0-8907-00c726179288 None admin] PATH_INFO: `/v3/system/users/b9530ded36ca4ae18cc3e7b07a3b7719/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:11.159085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f32439-c987-46b0-8907-00c726179288 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b9530ded36ca4ae18cc3e7b07a3b7719, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:11.161033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f32439-c987-46b0-8907-00c726179288 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:11.175714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 156/311] 10.222.0.27 () {58 vars in 1334 bytes} [Wed Jun 29 08:22:10 2022] PUT /identity/v3/system/users/b9530ded36ca4ae18cc3e7b07a3b7719/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:22:11.223128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 156/312] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:11.247729 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f32439-c987-46b0-8907-00c726179288 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:11.263627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:11.368753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f32439-c987-46b0-8907-00c726179288 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:11.406024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be85b-1471-4a2e-a73f-109c2ca78ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:11.406874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be85b-1471-4a2e-a73f-109c2ca78ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:11.407161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be85b-1471-4a2e-a73f-109c2ca78ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:11.407435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54be85b-1471-4a2e-a73f-109c2ca78ac5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:11.418857 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54be85b-1471-4a2e-a73f-109c2ca78ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a54be85b-1471-4a2e-a73f-109c2ca78ac5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:11.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:11.427700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:11.428484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 157/313] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:11.435115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fef25-45c0-4fa9-b43b-6a14478866e8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:11.475086 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:11.550256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fef25-45c0-4fa9-b43b-6a14478866e8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:11.585512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3eeac-f42a-4eea-9f73-725d8e2fdaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:11.586195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3eeac-f42a-4eea-9f73-725d8e2fdaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:11.586478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3eeac-f42a-4eea-9f73-725d8e2fdaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:11.586785 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3eeac-f42a-4eea-9f73-725d8e2fdaf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:11.587556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3eeac-f42a-4eea-9f73-725d8e2fdaf7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:11.587867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3eeac-f42a-4eea-9f73-725d8e2fdaf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:11.589587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3eeac-f42a-4eea-9f73-725d8e2fdaf7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:11.599395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 157/314] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:11.607808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be85b-1471-4a2e-a73f-109c2ca78ac5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:11.681629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:11.714973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54be85b-1471-4a2e-a73f-109c2ca78ac5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:11.751862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742eba7-1a99-490a-a13e-908669d6f47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:11.753426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742eba7-1a99-490a-a13e-908669d6f47a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:11.753945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742eba7-1a99-490a-a13e-908669d6f47a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:11.754479 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742eba7-1a99-490a-a13e-908669d6f47a None admin] PATH_INFO: `/v3/users/ee8c39cb257f45ad835ad20d7e005ae1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:11.755958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742eba7-1a99-490a-a13e-908669d6f47a None admin] RBAC: Authorizing `identity:get_user(user_id=ee8c39cb257f45ad835ad20d7e005ae1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:11.757598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742eba7-1a99-490a-a13e-908669d6f47a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:11.759776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 158/315] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:11 2022] GET /identity/v3/users/ee8c39cb257f45ad835ad20d7e005ae1 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:11.765423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3eeac-f42a-4eea-9f73-725d8e2fdaf7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:11.856297 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:11.866647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3eeac-f42a-4eea-9f73-725d8e2fdaf7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:11.901235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5bb8f-9d15-4f65-ae1d-dcf609109e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:11.901894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5bb8f-9d15-4f65-ae1d-dcf609109e93 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:11.902171 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5bb8f-9d15-4f65-ae1d-dcf609109e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:11.902445 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce5bb8f-9d15-4f65-ae1d-dcf609109e93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:11.975212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5bb8f-9d15-4f65-ae1d-dcf609109e93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:11.977054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce5bb8f-9d15-4f65-ae1d-dcf609109e93 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:12.071444 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:12.080581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 158/316] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:12.086946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742eba7-1a99-490a-a13e-908669d6f47a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:12.196448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742eba7-1a99-490a-a13e-908669d6f47a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:12.232927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41120e-9639-4db0-82dc-f9582dba0a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:12.233725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41120e-9639-4db0-82dc-f9582dba0a29 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:12.234015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41120e-9639-4db0-82dc-f9582dba0a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:12.234297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41120e-9639-4db0-82dc-f9582dba0a29 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:12.235530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41120e-9639-4db0-82dc-f9582dba0a29 None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:12.236986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41120e-9639-4db0-82dc-f9582dba0a29 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:12.239528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 159/317] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:12 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:12.245248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5bb8f-9d15-4f65-ae1d-dcf609109e93 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:12.355496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce5bb8f-9d15-4f65-ae1d-dcf609109e93 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:12.396248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592ffe7-6b2f-405d-98f3-74a651509027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:12.397290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592ffe7-6b2f-405d-98f3-74a651509027 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:12.397766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592ffe7-6b2f-405d-98f3-74a651509027 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:12.398230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592ffe7-6b2f-405d-98f3-74a651509027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:12.399332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592ffe7-6b2f-405d-98f3-74a651509027 None admin] RBAC: Adding query filter params (role.id=9a50f15db31f490294f370186ce2d7ff, scope.project.id=c862664ec560416a936d940af366502c, user.id=ee8c39cb257f45ad835ad20d7e005ae1) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:12.399843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592ffe7-6b2f-405d-98f3-74a651509027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:12.401465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592ffe7-6b2f-405d-98f3-74a651509027 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:12.408395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 159/318] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:12 2022] GET /identity/v3/role_assignments?user.id=ee8c39cb257f45ad835ad20d7e005ae1&scope.project.id=c862664ec560416a936d940af366502c&role.id=9a50f15db31f490294f370186ce2d7ff => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:12.662363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7644d0-1760-4e94-9ed9-006cd3b4936e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:12.662649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7644d0-1760-4e94-9ed9-006cd3b4936e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:12.662951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7644d0-1760-4e94-9ed9-006cd3b4936e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:12.664477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 160/319] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:12.669632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7164a3-3202-4146-b5b7-aad664552e6e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:12.669924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7164a3-3202-4146-b5b7-aad664552e6e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:12.670200 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7164a3-3202-4146-b5b7-aad664552e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:12.687014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dc7164a3-3202-4146-b5b7-aad664552e6e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:12.688192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dc7164a3-3202-4146-b5b7-aad664552e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:12.739219 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:12.759704 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 160/320] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:12.782232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7644d0-1760-4e94-9ed9-006cd3b4936e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:12.792621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:12.889543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7644d0-1760-4e94-9ed9-006cd3b4936e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:12.923312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408bfdd-dbde-4d2e-a860-c5785a39a688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:12.924027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408bfdd-dbde-4d2e-a860-c5785a39a688 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:12.924303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408bfdd-dbde-4d2e-a860-c5785a39a688 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:12.924570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408bfdd-dbde-4d2e-a860-c5785a39a688 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:12.926107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408bfdd-dbde-4d2e-a860-c5785a39a688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:12.927594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408bfdd-dbde-4d2e-a860-c5785a39a688 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:12.929372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 161/321] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:12.934998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7164a3-3202-4146-b5b7-aad664552e6e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:13.034662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7164a3-3202-4146-b5b7-aad664552e6e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:13.067981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe64f5-9c1c-412b-8e43-9427ad7e7379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:13.068653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe64f5-9c1c-412b-8e43-9427ad7e7379 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:13.068932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe64f5-9c1c-412b-8e43-9427ad7e7379 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:13.069203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe64f5-9c1c-412b-8e43-9427ad7e7379 None admin] PATH_INFO: `/v3/groups` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:13.069905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe64f5-9c1c-412b-8e43-9427ad7e7379 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:13.071371 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe64f5-9c1c-412b-8e43-9427ad7e7379 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:13.087594 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 161/322] 10.222.0.27 () {60 vars in 1134 bytes} [Wed Jun 29 08:22:12 2022] POST /identity/v3/groups => generated 226 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:22:13.305916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c3632-5f9d-455b-a989-0fb3c86c001c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:13.306416 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c3632-5f9d-455b-a989-0fb3c86c001c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:13.306416 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c3632-5f9d-455b-a989-0fb3c86c001c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:13.307817 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 162/323] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:13.313191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e616f-3c7b-414d-80ad-301119c09113 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:13.313492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e616f-3c7b-414d-80ad-301119c09113 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:13.313727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e616f-3c7b-414d-80ad-301119c09113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:13.328332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b45d99-9d25-49e5-8a33-ded6be3472a0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:13.328647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b45d99-9d25-49e5-8a33-ded6be3472a0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:13.328886 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b45d99-9d25-49e5-8a33-ded6be3472a0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:13.330233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 163/324] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:13.331751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-317e616f-3c7b-414d-80ad-301119c09113 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:13.332939 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-317e616f-3c7b-414d-80ad-301119c09113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:13.336099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df2ca-e4e3-4565-afc1-2434e741314e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:13.336455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df2ca-e4e3-4565-afc1-2434e741314e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:13.336799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df2ca-e4e3-4565-afc1-2434e741314e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:13.356493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2f9df2ca-e4e3-4565-afc1-2434e741314e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:13.357759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2f9df2ca-e4e3-4565-afc1-2434e741314e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:13.387715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:13.436588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 162/325] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:13.452531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 164/326] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:13.462092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e616f-3c7b-414d-80ad-301119c09113 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:13.481389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9df2ca-e4e3-4565-afc1-2434e741314e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:13.485978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:13.591402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e616f-3c7b-414d-80ad-301119c09113 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:13.624006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9df2ca-e4e3-4565-afc1-2434e741314e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:13.636584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77f0a7-8245-476a-a0c2-8efa681b1f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:13.637931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77f0a7-8245-476a-a0c2-8efa681b1f2f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:13.638319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77f0a7-8245-476a-a0c2-8efa681b1f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:13.638745 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77f0a7-8245-476a-a0c2-8efa681b1f2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:13.658825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77f0a7-8245-476a-a0c2-8efa681b1f2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ea77f0a7-8245-476a-a0c2-8efa681b1f2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:13.668060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:13.668988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:13.669775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 163/327] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:13.677145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77f0a7-8245-476a-a0c2-8efa681b1f2f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:13.694386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05083d31-11ce-43ac-8067-ca48c46bcada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:13.695810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05083d31-11ce-43ac-8067-ca48c46bcada None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:13.696486 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05083d31-11ce-43ac-8067-ca48c46bcada None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:13.696931 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05083d31-11ce-43ac-8067-ca48c46bcada None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:13.720527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05083d31-11ce-43ac-8067-ca48c46bcada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-05083d31-11ce-43ac-8067-ca48c46bcada None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:13.729386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:13.730307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:13.731128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:13.733927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 165/328] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:13.741600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:13.743636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05083d31-11ce-43ac-8067-ca48c46bcada None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:13.811106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77f0a7-8245-476a-a0c2-8efa681b1f2f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:13.819943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:13.849694 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7668e-1782-4e64-8559-fcc3ded7529a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:13.850742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7668e-1782-4e64-8559-fcc3ded7529a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:13.851243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7668e-1782-4e64-8559-fcc3ded7529a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:13.851715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7668e-1782-4e64-8559-fcc3ded7529a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:13.852684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7668e-1782-4e64-8559-fcc3ded7529a None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:13.853199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7668e-1782-4e64-8559-fcc3ded7529a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:13.854833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7668e-1782-4e64-8559-fcc3ded7529a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:13.864831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 164/329] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:13.869662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05083d31-11ce-43ac-8067-ca48c46bcada None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:13.870897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7668e-1782-4e64-8559-fcc3ded7529a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:13.903634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b22d9d-ac75-4ffd-8fe6-a16b4d68442f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:13.904347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22d9d-ac75-4ffd-8fe6-a16b4d68442f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:13.904590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22d9d-ac75-4ffd-8fe6-a16b4d68442f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:13.904867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22d9d-ac75-4ffd-8fe6-a16b4d68442f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:13.905649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b22d9d-ac75-4ffd-8fe6-a16b4d68442f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:13.906089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b22d9d-ac75-4ffd-8fe6-a16b4d68442f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:13.907397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b22d9d-ac75-4ffd-8fe6-a16b4d68442f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:13.917165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 166/330] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:13.926978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b22d9d-ac75-4ffd-8fe6-a16b4d68442f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:13.977552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:13.984457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7668e-1782-4e64-8559-fcc3ded7529a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:14.023311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834cc5c-b321-4782-8a2c-405f1fa3a637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:14.024456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834cc5c-b321-4782-8a2c-405f1fa3a637 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:14.024908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834cc5c-b321-4782-8a2c-405f1fa3a637 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:14.025326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834cc5c-b321-4782-8a2c-405f1fa3a637 None admin] PATH_INFO: `/v3/users/b9530ded36ca4ae18cc3e7b07a3b7719` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:14.026916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834cc5c-b321-4782-8a2c-405f1fa3a637 None admin] RBAC: Authorizing `identity:get_user(user_id=b9530ded36ca4ae18cc3e7b07a3b7719)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:14.028583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834cc5c-b321-4782-8a2c-405f1fa3a637 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:14.034817 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 165/331] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:13 2022] GET /identity/v3/users/b9530ded36ca4ae18cc3e7b07a3b7719 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:14.037169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834cc5c-b321-4782-8a2c-405f1fa3a637 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:14.052787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b22d9d-ac75-4ffd-8fe6-a16b4d68442f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:14.083085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:14.093546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82fa2f-6542-4b7f-a6b0-04b92ed05baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:14.094473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82fa2f-6542-4b7f-a6b0-04b92ed05baf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:14.094815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82fa2f-6542-4b7f-a6b0-04b92ed05baf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:14.095151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82fa2f-6542-4b7f-a6b0-04b92ed05baf None admin] PATH_INFO: `/v3/users/875e76dd3cb741248f910c395193e006` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:14.097119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82fa2f-6542-4b7f-a6b0-04b92ed05baf None admin] RBAC: Authorizing `identity:get_user(user_id=875e76dd3cb741248f910c395193e006)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:14.100552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82fa2f-6542-4b7f-a6b0-04b92ed05baf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:14.103612 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 167/332] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:13 2022] GET /identity/v3/users/875e76dd3cb741248f910c395193e006 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:14.111770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82fa2f-6542-4b7f-a6b0-04b92ed05baf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:14.164928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834cc5c-b321-4782-8a2c-405f1fa3a637 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:14.216838 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc0e67-a1ea-406d-b02f-cdd1217761eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:14.217565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0e67-a1ea-406d-b02f-cdd1217761eb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:14.217844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0e67-a1ea-406d-b02f-cdd1217761eb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:14.218492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0e67-a1ea-406d-b02f-cdd1217761eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:14.219105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0e67-a1ea-406d-b02f-cdd1217761eb None admin] RBAC: Adding query filter params (role.id=b0d377da1a7d4260be1f7ae0715fda91, user.id=b9530ded36ca4ae18cc3e7b07a3b7719, scope.system=all) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:14.219416 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0e67-a1ea-406d-b02f-cdd1217761eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:14.220869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0e67-a1ea-406d-b02f-cdd1217761eb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:14.235565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 166/333] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Jun 29 08:22:14 2022] GET /identity/v3/role_assignments?user.id=b9530ded36ca4ae18cc3e7b07a3b7719&scope.system=all&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:14.244808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82fa2f-6542-4b7f-a6b0-04b92ed05baf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:14.286078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39ebcd-5119-4a85-88bb-f746b7d700f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:14.287260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39ebcd-5119-4a85-88bb-f746b7d700f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:14.287720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39ebcd-5119-4a85-88bb-f746b7d700f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:14.288142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39ebcd-5119-4a85-88bb-f746b7d700f3 None admin] PATH_INFO: `/v3/system/users/875e76dd3cb741248f910c395193e006/roles/9a50f15db31f490294f370186ce2d7ff` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:14.289919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39ebcd-5119-4a85-88bb-f746b7d700f3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=875e76dd3cb741248f910c395193e006, role_id=9a50f15db31f490294f370186ce2d7ff)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:14.291568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39ebcd-5119-4a85-88bb-f746b7d700f3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:14.304036 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 168/334] 10.222.0.27 () {58 vars in 1334 bytes} [Wed Jun 29 08:22:14 2022] PUT /identity/v3/system/users/875e76dd3cb741248f910c395193e006/roles/9a50f15db31f490294f370186ce2d7ff => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:15.605551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78ccb8-3783-49b1-8391-b2020efb186a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:15.606070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78ccb8-3783-49b1-8391-b2020efb186a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:15.606070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78ccb8-3783-49b1-8391-b2020efb186a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:15.607761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 167/335] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:15.613458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a64bdc-fbbe-4efa-b1c6-f5305b2494aa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:15.614290 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a64bdc-fbbe-4efa-b1c6-f5305b2494aa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:15.614795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a64bdc-fbbe-4efa-b1c6-f5305b2494aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:15.639732 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e1a64bdc-fbbe-4efa-b1c6-f5305b2494aa None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:15.641332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e1a64bdc-fbbe-4efa-b1c6-f5305b2494aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:15.711957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:15.736954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 169/336] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:15.780863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78ccb8-3783-49b1-8391-b2020efb186a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:15.816312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:15.908941 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78ccb8-3783-49b1-8391-b2020efb186a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:15.956606 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca81bc-716c-4e5f-bc8e-d225de3cf517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:15.957418 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca81bc-716c-4e5f-bc8e-d225de3cf517 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:15.957699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca81bc-716c-4e5f-bc8e-d225de3cf517 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:15.957969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca81bc-716c-4e5f-bc8e-d225de3cf517 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:15.970502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca81bc-716c-4e5f-bc8e-d225de3cf517 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-74ca81bc-716c-4e5f-bc8e-d225de3cf517 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:15.978485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:15.979386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:15.980166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 168/337] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:15.985643 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a64bdc-fbbe-4efa-b1c6-f5305b2494aa None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:16.127657 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a64bdc-fbbe-4efa-b1c6-f5305b2494aa None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:16.170648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e902375-4723-4564-a6ca-710614ad1f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:16.171361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902375-4723-4564-a6ca-710614ad1f0f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:16.171676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902375-4723-4564-a6ca-710614ad1f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:16.171995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902375-4723-4564-a6ca-710614ad1f0f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:16.172766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902375-4723-4564-a6ca-710614ad1f0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:16.173079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902375-4723-4564-a6ca-710614ad1f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:16.174573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902375-4723-4564-a6ca-710614ad1f0f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:16.184813 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 170/338] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:16.190468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca81bc-716c-4e5f-bc8e-d225de3cf517 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:16.307111 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:16.326455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ead70d-e8f0-49e9-81a0-9de18a1b885f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:16.327133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ead70d-e8f0-49e9-81a0-9de18a1b885f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:16.327607 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ead70d-e8f0-49e9-81a0-9de18a1b885f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:16.329701 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 171/339] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:16.333265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca81bc-716c-4e5f-bc8e-d225de3cf517 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:16.335002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35e88e-ec06-4cf1-9168-5fe0d22a1fbf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:16.335517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35e88e-ec06-4cf1-9168-5fe0d22a1fbf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:16.335990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35e88e-ec06-4cf1-9168-5fe0d22a1fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:16.370951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e14f73-5285-4e0b-baa4-c4157d48d0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:16.371612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e14f73-5285-4e0b-baa4-c4157d48d0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:16.371893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e14f73-5285-4e0b-baa4-c4157d48d0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:16.372162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e14f73-5285-4e0b-baa4-c4157d48d0b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:16.372788 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ed35e88e-ec06-4cf1-9168-5fe0d22a1fbf None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:16.374197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ed35e88e-ec06-4cf1-9168-5fe0d22a1fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:16.449288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e14f73-5285-4e0b-baa4-c4157d48d0b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:16.453806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e14f73-5285-4e0b-baa4-c4157d48d0b3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:16.493974 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:16.498242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 172/340] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:16.536194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:16.538741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35e88e-ec06-4cf1-9168-5fe0d22a1fbf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:16.558939 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 169/341] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:16.560800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e14f73-5285-4e0b-baa4-c4157d48d0b3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:16.661739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e14f73-5285-4e0b-baa4-c4157d48d0b3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:16.694183 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35e88e-ec06-4cf1-9168-5fe0d22a1fbf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:16.697641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4690c-6134-4053-8f51-e5065d1f0af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:16.698790 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4690c-6134-4053-8f51-e5065d1f0af3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:16.699290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4690c-6134-4053-8f51-e5065d1f0af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:16.699748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4690c-6134-4053-8f51-e5065d1f0af3 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:16.701143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4690c-6134-4053-8f51-e5065d1f0af3 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:16.702804 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4690c-6134-4053-8f51-e5065d1f0af3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:16.704719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 170/342] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:16 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:16.710664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4690c-6134-4053-8f51-e5065d1f0af3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:16.753263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7397bc65-3cd9-4e3b-aff0-88a4117b4d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:16.754497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397bc65-3cd9-4e3b-aff0-88a4117b4d19 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:16.754985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397bc65-3cd9-4e3b-aff0-88a4117b4d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:16.755439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397bc65-3cd9-4e3b-aff0-88a4117b4d19 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:16.781310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7397bc65-3cd9-4e3b-aff0-88a4117b4d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7397bc65-3cd9-4e3b-aff0-88a4117b4d19 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:16.789844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:16.790726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:16.791507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:16.793900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 173/343] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:22:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:16.803575 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200b70d-5a70-4856-8b90-4d36cf349979 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:16.808215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200b70d-5a70-4856-8b90-4d36cf349979 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:16.808698 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200b70d-5a70-4856-8b90-4d36cf349979 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:16.810359 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 174/344] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:16 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:16.810830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:16.812841 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200b70d-5a70-4856-8b90-4d36cf349979 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:16.899319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4690c-6134-4053-8f51-e5065d1f0af3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:16.901610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:16.971451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a427557-7ac5-4ef8-b2e1-45bd032d7e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:16.972905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a427557-7ac5-4ef8-b2e1-45bd032d7e0d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:16.973401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a427557-7ac5-4ef8-b2e1-45bd032d7e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:16.973860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a427557-7ac5-4ef8-b2e1-45bd032d7e0d None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:16.985104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a427557-7ac5-4ef8-b2e1-45bd032d7e0d None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:16.987642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a427557-7ac5-4ef8-b2e1-45bd032d7e0d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:16.998065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 171/345] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:16 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:16.999692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6448d6b-a5d5-45f7-b9dd-a2df7df888f4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.000345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6448d6b-a5d5-45f7-b9dd-a2df7df888f4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.002094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6448d6b-a5d5-45f7-b9dd-a2df7df888f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.027606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200b70d-5a70-4856-8b90-4d36cf349979 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:17.035875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e6448d6b-a5d5-45f7-b9dd-a2df7df888f4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:17.037481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e6448d6b-a5d5-45f7-b9dd-a2df7df888f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:17.085585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fd03b-f48d-4dc9-b5e6-07d7ad14ba70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:17.086249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fd03b-f48d-4dc9-b5e6-07d7ad14ba70 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.086534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fd03b-f48d-4dc9-b5e6-07d7ad14ba70 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.087003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fd03b-f48d-4dc9-b5e6-07d7ad14ba70 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.087937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fd03b-f48d-4dc9-b5e6-07d7ad14ba70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:17.088594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fd03b-f48d-4dc9-b5e6-07d7ad14ba70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:17.090668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fd03b-f48d-4dc9-b5e6-07d7ad14ba70 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:17.107595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 175/346] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:17.108492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fd03b-f48d-4dc9-b5e6-07d7ad14ba70 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:17.133502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 172/347] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:17.134881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6448d6b-a5d5-45f7-b9dd-a2df7df888f4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:17.191095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:17.246433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6448d6b-a5d5-45f7-b9dd-a2df7df888f4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:17.284601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba46f76-a5ca-463d-9ab8-d5f83dc2b1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:17.285708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba46f76-a5ca-463d-9ab8-d5f83dc2b1ca None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.286177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba46f76-a5ca-463d-9ab8-d5f83dc2b1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.286682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba46f76-a5ca-463d-9ab8-d5f83dc2b1ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.317065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fd03b-f48d-4dc9-b5e6-07d7ad14ba70 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:17.318772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:17.371080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645bc9e4-ea76-46cd-be18-f0048584099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:17.371667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bc9e4-ea76-46cd-be18-f0048584099e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.372006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bc9e4-ea76-46cd-be18-f0048584099e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.372290 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bc9e4-ea76-46cd-be18-f0048584099e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.373147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bc9e4-ea76-46cd-be18-f0048584099e None admin] RBAC: Adding query filter params (group.id=a2395cbf5c22469f90759ba53cedbdb5, role.id=b0d377da1a7d4260be1f7ae0715fda91, scope.project.id=314e1f70a6da40a79963abc77a1351c4) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:17.373462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bc9e4-ea76-46cd-be18-f0048584099e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:17.375819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bc9e4-ea76-46cd-be18-f0048584099e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:17.377698 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba46f76-a5ca-463d-9ab8-d5f83dc2b1ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:17.379587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba46f76-a5ca-463d-9ab8-d5f83dc2b1ca None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:17.386731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 176/348] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jun 29 08:22:17 2022] GET /identity/v3/role_assignments?group.id=a2395cbf5c22469f90759ba53cedbdb5&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 260 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:17.386731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645bc9e4-ea76-46cd-be18-f0048584099e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:17.522937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:17.525319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645bc9e4-ea76-46cd-be18-f0048584099e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:17.568615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76299d-05bf-4a38-9b4f-f2a1a5f22a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:17.569735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76299d-05bf-4a38-9b4f-f2a1a5f22a71 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.570187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76299d-05bf-4a38-9b4f-f2a1a5f22a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.570626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76299d-05bf-4a38-9b4f-f2a1a5f22a71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.584059 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76299d-05bf-4a38-9b4f-f2a1a5f22a71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3c76299d-05bf-4a38-9b4f-f2a1a5f22a71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:17.592836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:17.593737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:17.594534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:17.597184 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 177/349] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:17.598774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe5cf1-9996-4ea4-9d1c-27430fc0a021 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.599201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe5cf1-9996-4ea4-9d1c-27430fc0a021 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.599616 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe5cf1-9996-4ea4-9d1c-27430fc0a021 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.601298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 178/350] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:17.602528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf354f-b5c6-4469-94ac-4b7120a18b43 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.606682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 173/351] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:17.607314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba46f76-a5ca-463d-9ab8-d5f83dc2b1ca None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:17.607962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf354f-b5c6-4469-94ac-4b7120a18b43 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.608426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf354f-b5c6-4469-94ac-4b7120a18b43 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.610118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 179/352] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:17.611589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaf85d-0787-4745-a094-f9764da123f4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.612046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaf85d-0787-4745-a094-f9764da123f4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.612740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdaf85d-0787-4745-a094-f9764da123f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.635620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0fdaf85d-0787-4745-a094-f9764da123f4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:17.637202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0fdaf85d-0787-4745-a094-f9764da123f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:17.660932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:17.712392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 180/353] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:17.714000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba46f76-a5ca-463d-9ab8-d5f83dc2b1ca None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:17.715242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55998a-a6ca-4244-964e-4c2c5903535c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.715755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55998a-a6ca-4244-964e-4c2c5903535c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.716265 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55998a-a6ca-4244-964e-4c2c5903535c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.733891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-df55998a-a6ca-4244-964e-4c2c5903535c None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:17.735493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-df55998a-a6ca-4244-964e-4c2c5903535c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:17.754940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317920a8-5d72-456a-998e-b4e53d518210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:17.755736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317920a8-5d72-456a-998e-b4e53d518210 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.756086 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317920a8-5d72-456a-998e-b4e53d518210 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.756421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317920a8-5d72-456a-998e-b4e53d518210 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.757249 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317920a8-5d72-456a-998e-b4e53d518210 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:17.757612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317920a8-5d72-456a-998e-b4e53d518210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:17.759315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317920a8-5d72-456a-998e-b4e53d518210 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:17.769314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 174/354] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:17.770080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317920a8-5d72-456a-998e-b4e53d518210 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:17.779937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:17.809991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 181/355] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:17.811168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55998a-a6ca-4244-964e-4c2c5903535c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:17.877039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317920a8-5d72-456a-998e-b4e53d518210 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:17.902425 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:17.914861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fac07-5e96-45be-8741-ecc2425a0d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:17.915858 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fac07-5e96-45be-8741-ecc2425a0d10 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.916203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fac07-5e96-45be-8741-ecc2425a0d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.916540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fac07-5e96-45be-8741-ecc2425a0d10 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.917997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fac07-5e96-45be-8741-ecc2425a0d10 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:17.919591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5fac07-5e96-45be-8741-ecc2425a0d10 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:17.921397 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 175/356] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:17 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:17.922280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fac07-5e96-45be-8741-ecc2425a0d10 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:17.922727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55998a-a6ca-4244-964e-4c2c5903535c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:17.965304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69ebc-3447-48ec-81d4-fe02afe84dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:17.966429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69ebc-3447-48ec-81d4-fe02afe84dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:17.966946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69ebc-3447-48ec-81d4-fe02afe84dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:17.967398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d69ebc-3447-48ec-81d4-fe02afe84dd0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:17.979040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d69ebc-3447-48ec-81d4-fe02afe84dd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-06d69ebc-3447-48ec-81d4-fe02afe84dd0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:17.987465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:17.988335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:17.989111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:17.991897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 182/357] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:22:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:17.992996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69ebc-3447-48ec-81d4-fe02afe84dd0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:18.015065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:18.039135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fac07-5e96-45be-8741-ecc2425a0d10 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:18.077918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f63c1f-dc7f-430e-9fa0-86840d78ec61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:18.078584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f63c1f-dc7f-430e-9fa0-86840d78ec61 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:18.078949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f63c1f-dc7f-430e-9fa0-86840d78ec61 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:18.079224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f63c1f-dc7f-430e-9fa0-86840d78ec61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:18.106040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d69ebc-3447-48ec-81d4-fe02afe84dd0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:18.142667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120707eb-d259-4310-8676-70c9cf4fe1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:18.143951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120707eb-d259-4310-8676-70c9cf4fe1f8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:18.144510 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120707eb-d259-4310-8676-70c9cf4fe1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:18.145024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120707eb-d259-4310-8676-70c9cf4fe1f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:18.157024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120707eb-d259-4310-8676-70c9cf4fe1f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:18.157183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f63c1f-dc7f-430e-9fa0-86840d78ec61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:18.158542 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f63c1f-dc7f-430e-9fa0-86840d78ec61 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-120707eb-d259-4310-8676-70c9cf4fe1f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:18.166369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:18.167589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 08:22:18.168458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:18.170897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 183/358] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:18.172120 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120707eb-d259-4310-8676-70c9cf4fe1f8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:18.183506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:18.247267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:18.277391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 176/359] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:18.278681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f63c1f-dc7f-430e-9fa0-86840d78ec61 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:18.301877 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120707eb-d259-4310-8676-70c9cf4fe1f8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:18.344531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3f5d9-3ccb-4faf-997b-b9ddd0ae7416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:18.345399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3f5d9-3ccb-4faf-997b-b9ddd0ae7416 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:18.345674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3f5d9-3ccb-4faf-997b-b9ddd0ae7416 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:18.345943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3f5d9-3ccb-4faf-997b-b9ddd0ae7416 None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:18.348463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3f5d9-3ccb-4faf-997b-b9ddd0ae7416 None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:18.349920 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3f5d9-3ccb-4faf-997b-b9ddd0ae7416 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:18.351641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 184/360] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:18 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:18.352474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3f5d9-3ccb-4faf-997b-b9ddd0ae7416 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:18.406873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f63c1f-dc7f-430e-9fa0-86840d78ec61 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:18.442616 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:18.452859 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2333c3-3692-4e1e-941a-71d91fd4030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:18.453542 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2333c3-3692-4e1e-941a-71d91fd4030b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:18.453820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2333c3-3692-4e1e-941a-71d91fd4030b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:18.454091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2333c3-3692-4e1e-941a-71d91fd4030b None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:18.455846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2333c3-3692-4e1e-941a-71d91fd4030b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:18.455846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2333c3-3692-4e1e-941a-71d91fd4030b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:18.456687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2333c3-3692-4e1e-941a-71d91fd4030b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:18.466738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 177/361] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:18.466738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2333c3-3692-4e1e-941a-71d91fd4030b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:18.489015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3f5d9-3ccb-4faf-997b-b9ddd0ae7416 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:18.509753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:18.534651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0977c16-c61a-4d73-b6de-102f305bd519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:18.535382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0977c16-c61a-4d73-b6de-102f305bd519 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:18.535663 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0977c16-c61a-4d73-b6de-102f305bd519 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:18.535936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0977c16-c61a-4d73-b6de-102f305bd519 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:18.536692 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0977c16-c61a-4d73-b6de-102f305bd519 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:18.537001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0977c16-c61a-4d73-b6de-102f305bd519 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:18.539631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0977c16-c61a-4d73-b6de-102f305bd519 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:18.549350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 185/362] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:18.551020 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c82088-3f95-44e5-a287-e5a171d7656e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:18.551307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c82088-3f95-44e5-a287-e5a171d7656e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:18.551585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c82088-3f95-44e5-a287-e5a171d7656e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:18.553078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 186/363] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:18.553832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c82088-3f95-44e5-a287-e5a171d7656e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:18.600522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2333c3-3692-4e1e-941a-71d91fd4030b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:18.641021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33ef26-5176-4ea0-801a-153886487056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:18.642012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33ef26-5176-4ea0-801a-153886487056 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:18.642291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33ef26-5176-4ea0-801a-153886487056 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:18.642568 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33ef26-5176-4ea0-801a-153886487056 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:18.643874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33ef26-5176-4ea0-801a-153886487056 None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:18.645439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33ef26-5176-4ea0-801a-153886487056 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:18.647516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 178/364] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:18 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:18.648093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33ef26-5176-4ea0-801a-153886487056 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:18.689052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c82088-3f95-44e5-a287-e5a171d7656e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:18.752891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd6413a-0429-4db3-9007-9fd1c6027ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:18.754532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6413a-0429-4db3-9007-9fd1c6027ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:18.755076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6413a-0429-4db3-9007-9fd1c6027ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:18.755984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd6413a-0429-4db3-9007-9fd1c6027ba0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:18.757343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6413a-0429-4db3-9007-9fd1c6027ba0 None admin] RBAC: Adding query filter params (group.id=a2395cbf5c22469f90759ba53cedbdb5, role.id=3531cd7306964cc5967f92c559890ae5, scope.project.id=314e1f70a6da40a79963abc77a1351c4) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:18.757854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6413a-0429-4db3-9007-9fd1c6027ba0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:18.760234 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd6413a-0429-4db3-9007-9fd1c6027ba0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:18.765892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:18.769175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 187/365] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jun 29 08:22:18 2022] GET /identity/v3/role_assignments?group.id=a2395cbf5c22469f90759ba53cedbdb5&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=3531cd7306964cc5967f92c559890ae5 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:18.773914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08412971-4790-44a7-b11d-08a375a52d43 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:18.775122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08412971-4790-44a7-b11d-08a375a52d43 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:18.775808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08412971-4790-44a7-b11d-08a375a52d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:18.784098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33ef26-5176-4ea0-801a-153886487056 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:18.799088 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:18.810552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-08412971-4790-44a7-b11d-08a375a52d43 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:18.811821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-08412971-4790-44a7-b11d-08a375a52d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:18.832512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a26599-6304-41ba-a794-2e4d31f12744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:18.833188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a26599-6304-41ba-a794-2e4d31f12744 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:18.833501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a26599-6304-41ba-a794-2e4d31f12744 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:18.833753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a26599-6304-41ba-a794-2e4d31f12744 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:18.914560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a26599-6304-41ba-a794-2e4d31f12744 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:18.916267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a26599-6304-41ba-a794-2e4d31f12744 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:18.920807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 188/366] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:18.921979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08412971-4790-44a7-b11d-08a375a52d43 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:19.033307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08412971-4790-44a7-b11d-08a375a52d43 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:19.039482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 179/367] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:19.040355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a26599-6304-41ba-a794-2e4d31f12744 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:19.043547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:19.067534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:19.074063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343ee69-fdb5-4384-943f-6b2b0c22cc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:19.075280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343ee69-fdb5-4384-943f-6b2b0c22cc90 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:19.075808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343ee69-fdb5-4384-943f-6b2b0c22cc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:19.076440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343ee69-fdb5-4384-943f-6b2b0c22cc90 None admin] PATH_INFO: `/v3/users/875e76dd3cb741248f910c395193e006` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:19.077920 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343ee69-fdb5-4384-943f-6b2b0c22cc90 None admin] RBAC: Authorizing `identity:get_user(user_id=875e76dd3cb741248f910c395193e006)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:19.079546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343ee69-fdb5-4384-943f-6b2b0c22cc90 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:19.081744 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 189/368] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:22:18 2022] GET /identity/v3/users/875e76dd3cb741248f910c395193e006 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:19.082648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343ee69-fdb5-4384-943f-6b2b0c22cc90 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:19.158010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a26599-6304-41ba-a794-2e4d31f12744 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:19.194573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343ee69-fdb5-4384-943f-6b2b0c22cc90 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:19.195162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157453c2-bc61-42ee-85aa-e8e459fcf559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:19.196217 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157453c2-bc61-42ee-85aa-e8e459fcf559 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:19.196628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157453c2-bc61-42ee-85aa-e8e459fcf559 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:19.197031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157453c2-bc61-42ee-85aa-e8e459fcf559 None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:19.198357 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157453c2-bc61-42ee-85aa-e8e459fcf559 None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:19.199977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157453c2-bc61-42ee-85aa-e8e459fcf559 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:19.201779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 180/369] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:19 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:19.202672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157453c2-bc61-42ee-85aa-e8e459fcf559 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:19.233308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8d7f8-2771-49da-b138-b631bdafe6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:19.234307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8d7f8-2771-49da-b138-b631bdafe6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:19.234785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8d7f8-2771-49da-b138-b631bdafe6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:19.235162 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8d7f8-2771-49da-b138-b631bdafe6e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:19.246554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8d7f8-2771-49da-b138-b631bdafe6e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8bf8d7f8-2771-49da-b138-b631bdafe6e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:19.254810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:19.255772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:19.256540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:19.257122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 190/370] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:22:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:19.258081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8d7f8-2771-49da-b138-b631bdafe6e8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:19.280922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:19.313255 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157453c2-bc61-42ee-85aa-e8e459fcf559 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:19.321842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:19.350520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb1637-0e4b-4d4c-bc6a-3ce5d7a55de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:19.351810 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb1637-0e4b-4d4c-bc6a-3ce5d7a55de4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:19.352300 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb1637-0e4b-4d4c-bc6a-3ce5d7a55de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:19.352906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb1637-0e4b-4d4c-bc6a-3ce5d7a55de4 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:19.354280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb1637-0e4b-4d4c-bc6a-3ce5d7a55de4 None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:19.355949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bb1637-0e4b-4d4c-bc6a-3ce5d7a55de4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:19.357834 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 181/371] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:19 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:19.358823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb1637-0e4b-4d4c-bc6a-3ce5d7a55de4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:19.369710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8d7f8-2771-49da-b138-b631bdafe6e8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:19.406409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6b078-83d7-4ef5-afeb-e9ee11d99acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:19.407151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6b078-83d7-4ef5-afeb-e9ee11d99acf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:19.407446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6b078-83d7-4ef5-afeb-e9ee11d99acf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:19.407734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6b078-83d7-4ef5-afeb-e9ee11d99acf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:19.408787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6b078-83d7-4ef5-afeb-e9ee11d99acf None admin] RBAC: Adding query filter params (role.id=9a50f15db31f490294f370186ce2d7ff, user.id=875e76dd3cb741248f910c395193e006, scope.system=all) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:19.408894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6b078-83d7-4ef5-afeb-e9ee11d99acf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:19.410565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6b078-83d7-4ef5-afeb-e9ee11d99acf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:19.423618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 191/372] 10.222.0.27 () {56 vars in 1308 bytes} [Wed Jun 29 08:22:19 2022] GET /identity/v3/role_assignments?user.id=875e76dd3cb741248f910c395193e006&scope.system=all&role.id=9a50f15db31f490294f370186ce2d7ff => generated 510 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:19.424463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6b078-83d7-4ef5-afeb-e9ee11d99acf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:19.480822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb1637-0e4b-4d4c-bc6a-3ce5d7a55de4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:19.517970 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc8906-41ce-41ae-ae56-62beaac481e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:19.519115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc8906-41ce-41ae-ae56-62beaac481e1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:19.519628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc8906-41ce-41ae-ae56-62beaac481e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:19.520094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc8906-41ce-41ae-ae56-62beaac481e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:19.521159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc8906-41ce-41ae-ae56-62beaac481e1 None admin] RBAC: Adding query filter params (group.id=a2395cbf5c22469f90759ba53cedbdb5, role.id=b0d377da1a7d4260be1f7ae0715fda91, scope.project.id=c862664ec560416a936d940af366502c) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:19.521679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc8906-41ce-41ae-ae56-62beaac481e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:19.523407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc8906-41ce-41ae-ae56-62beaac481e1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:19.530201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 182/373] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jun 29 08:22:19 2022] GET /identity/v3/role_assignments?group.id=a2395cbf5c22469f90759ba53cedbdb5&scope.project.id=c862664ec560416a936d940af366502c&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:19.531314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc8906-41ce-41ae-ae56-62beaac481e1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:19.535524 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:19.540237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6b078-83d7-4ef5-afeb-e9ee11d99acf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:19.559468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:19.580118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f94823-74b4-4208-9b51-f7495428de6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:19.580726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f94823-74b4-4208-9b51-f7495428de6f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:19.581235 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f94823-74b4-4208-9b51-f7495428de6f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:19.581519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f94823-74b4-4208-9b51-f7495428de6f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:19.582320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f94823-74b4-4208-9b51-f7495428de6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:19.582637 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f94823-74b4-4208-9b51-f7495428de6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:19.584169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f94823-74b4-4208-9b51-f7495428de6f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:19.594834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 192/374] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:19.600524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f94823-74b4-4208-9b51-f7495428de6f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:19.669501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc8906-41ce-41ae-ae56-62beaac481e1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:19.725182 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3efb21-0767-4220-ba4f-54e20edeef08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:19.726781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3efb21-0767-4220-ba4f-54e20edeef08 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:19.728112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3efb21-0767-4220-ba4f-54e20edeef08 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:19.728823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3efb21-0767-4220-ba4f-54e20edeef08 None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:19.733368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3efb21-0767-4220-ba4f-54e20edeef08 None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:19.735109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3efb21-0767-4220-ba4f-54e20edeef08 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:19.737536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 183/375] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:19 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:19.743768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3efb21-0767-4220-ba4f-54e20edeef08 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:19.760139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f94823-74b4-4208-9b51-f7495428de6f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:19.802794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06f376-c9d1-4834-9bc6-0e0fa0c6f6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:19.803751 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06f376-c9d1-4834-9bc6-0e0fa0c6f6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:19.804162 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06f376-c9d1-4834-9bc6-0e0fa0c6f6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:19.804560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06f376-c9d1-4834-9bc6-0e0fa0c6f6c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:19.860400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:19.884373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06f376-c9d1-4834-9bc6-0e0fa0c6f6c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:19.886075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06f376-c9d1-4834-9bc6-0e0fa0c6f6c7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:19.894766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:19.931764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3efb21-0767-4220-ba4f-54e20edeef08 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:19.978491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e971c34-e7d0-45f3-8abf-5983b390c7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:19.979356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e971c34-e7d0-45f3-8abf-5983b390c7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:19.979672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e971c34-e7d0-45f3-8abf-5983b390c7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:19.980012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e971c34-e7d0-45f3-8abf-5983b390c7a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:19.980874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e971c34-e7d0-45f3-8abf-5983b390c7a1 None admin] RBAC: Adding query filter params (group.id=a2395cbf5c22469f90759ba53cedbdb5, role.id=3531cd7306964cc5967f92c559890ae5, scope.project.id=c862664ec560416a936d940af366502c) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:19.981190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e971c34-e7d0-45f3-8abf-5983b390c7a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:19.983794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e971c34-e7d0-45f3-8abf-5983b390c7a1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:19.989214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 193/376] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 08:22:19.992120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 184/377] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jun 29 08:22:19 2022] GET /identity/v3/role_assignments?group.id=a2395cbf5c22469f90759ba53cedbdb5&scope.project.id=c862664ec560416a936d940af366502c&role.id=3531cd7306964cc5967f92c559890ae5 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:20.001945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06f376-c9d1-4834-9bc6-0e0fa0c6f6c7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:20.104582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611245b-5a43-4b18-bce5-9a97d22c28f2 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:20.104874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611245b-5a43-4b18-bce5-9a97d22c28f2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:20.105130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611245b-5a43-4b18-bce5-9a97d22c28f2 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:20.106517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 185/378] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:22:20.112264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c14f67-c8ab-46c1-b776-b1ff1a2746a2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:20.112539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c14f67-c8ab-46c1-b776-b1ff1a2746a2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:20.112813 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c14f67-c8ab-46c1-b776-b1ff1a2746a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:20.114410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:20.141492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-17c14f67-c8ab-46c1-b776-b1ff1a2746a2 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:20.143047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-17c14f67-c8ab-46c1-b776-b1ff1a2746a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:20.152987 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06f376-c9d1-4834-9bc6-0e0fa0c6f6c7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:20.217309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6102fec-ffa9-4e60-8832-98b21f63deae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:20.218519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6102fec-ffa9-4e60-8832-98b21f63deae None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:20.219018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6102fec-ffa9-4e60-8832-98b21f63deae None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:20.219461 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6102fec-ffa9-4e60-8832-98b21f63deae None admin] PATH_INFO: `/v3/projects/3b8a36f8c21e4c4f9394d7b2d795e615` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:20.221885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6102fec-ffa9-4e60-8832-98b21f63deae None admin] RBAC: Authorizing `identity:get_project(project_id=3b8a36f8c21e4c4f9394d7b2d795e615)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:20.224852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6102fec-ffa9-4e60-8832-98b21f63deae None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:20.229942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 194/379] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:20 2022] GET /identity/v3/projects/3b8a36f8c21e4c4f9394d7b2d795e615 => generated 344 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:20.234239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6102fec-ffa9-4e60-8832-98b21f63deae None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:20.258795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 186/380] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:20.277953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c14f67-c8ab-46c1-b776-b1ff1a2746a2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:20.356249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6102fec-ffa9-4e60-8832-98b21f63deae None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:20.369198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:20.400880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904dcd98-d467-4845-be7b-c36fd5532826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:20.402126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dcd98-d467-4845-be7b-c36fd5532826 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:20.402561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dcd98-d467-4845-be7b-c36fd5532826 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:20.403107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dcd98-d467-4845-be7b-c36fd5532826 None admin] PATH_INFO: `/v3/groups/04745421c66d446499bbc4253685cb47` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:20.408721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c14f67-c8ab-46c1-b776-b1ff1a2746a2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:20.410551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904dcd98-d467-4845-be7b-c36fd5532826 None admin] RBAC: Authorizing `identity:get_group(group_id=04745421c66d446499bbc4253685cb47)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:20.412322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904dcd98-d467-4845-be7b-c36fd5532826 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:20.414566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 195/381] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:20 2022] GET /identity/v3/groups/04745421c66d446499bbc4253685cb47 => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:20.419959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904dcd98-d467-4845-be7b-c36fd5532826 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:20.435037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:20.483443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133d692-be39-4efd-87e0-bae552bbfca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:20.484274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133d692-be39-4efd-87e0-bae552bbfca1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:20.484557 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133d692-be39-4efd-87e0-bae552bbfca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:20.484827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133d692-be39-4efd-87e0-bae552bbfca1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:20.508278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133d692-be39-4efd-87e0-bae552bbfca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-9133d692-be39-4efd-87e0-bae552bbfca1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:20.520181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:20.521140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:20.521954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 187/382] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:20.527449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133d692-be39-4efd-87e0-bae552bbfca1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:20.534399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904dcd98-d467-4845-be7b-c36fd5532826 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:20.578711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e170f8e-d5ae-451c-ac14-9f2856432c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:20.579443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e170f8e-d5ae-451c-ac14-9f2856432c22 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:20.579679 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e170f8e-d5ae-451c-ac14-9f2856432c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:20.579958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e170f8e-d5ae-451c-ac14-9f2856432c22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:20.581035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e170f8e-d5ae-451c-ac14-9f2856432c22 None admin] RBAC: Adding query filter params (group.id=04745421c66d446499bbc4253685cb47, role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=3b8a36f8c21e4c4f9394d7b2d795e615) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:20.581356 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e170f8e-d5ae-451c-ac14-9f2856432c22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:20.583029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e170f8e-d5ae-451c-ac14-9f2856432c22 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:20.594966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 196/383] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jun 29 08:22:20 2022] GET /identity/v3/role_assignments?group.id=04745421c66d446499bbc4253685cb47&scope.project.id=3b8a36f8c21e4c4f9394d7b2d795e615&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:20.650271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133d692-be39-4efd-87e0-bae552bbfca1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:20.686037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fe8f9-557e-41f0-aadb-17a6ce495506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:20.686758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fe8f9-557e-41f0-aadb-17a6ce495506 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:20.687195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:20.687696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fe8f9-557e-41f0-aadb-17a6ce495506 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:20.688150 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fe8f9-557e-41f0-aadb-17a6ce495506 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:20.689110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fe8f9-557e-41f0-aadb-17a6ce495506 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:20.689603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fe8f9-557e-41f0-aadb-17a6ce495506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:20.691569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47fe8f9-557e-41f0-aadb-17a6ce495506 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:20.702240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 188/384] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:20.707761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e170f8e-d5ae-451c-ac14-9f2856432c22 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:20.815829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:20.852691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e170f8e-d5ae-451c-ac14-9f2856432c22 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:20.894175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65b050-16c9-48c6-9667-d5e204d8a244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:20.895072 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65b050-16c9-48c6-9667-d5e204d8a244 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:20.895355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65b050-16c9-48c6-9667-d5e204d8a244 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:20.895629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65b050-16c9-48c6-9667-d5e204d8a244 None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:20.896842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65b050-16c9-48c6-9667-d5e204d8a244 None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:20.898306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65b050-16c9-48c6-9667-d5e204d8a244 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:20.901344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 197/385] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:20 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:20.905950 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fe8f9-557e-41f0-aadb-17a6ce495506 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:21.039222 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47fe8f9-557e-41f0-aadb-17a6ce495506 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:21.074976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572051f-9e02-4de9-bf03-b3642f381a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:21.075852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572051f-9e02-4de9-bf03-b3642f381a21 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:21.076129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572051f-9e02-4de9-bf03-b3642f381a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:21.076397 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572051f-9e02-4de9-bf03-b3642f381a21 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:21.077579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572051f-9e02-4de9-bf03-b3642f381a21 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:21.079042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f572051f-9e02-4de9-bf03-b3642f381a21 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:21.081111 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 189/386] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:20 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:21.086934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65b050-16c9-48c6-9667-d5e204d8a244 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:21.254520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65b050-16c9-48c6-9667-d5e204d8a244 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:21.300649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4cfda-5d43-4844-885d-66121299d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:21.301542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4cfda-5d43-4844-885d-66121299d412 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:21.301821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4cfda-5d43-4844-885d-66121299d412 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:21.302096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4cfda-5d43-4844-885d-66121299d412 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4/groups/a2395cbf5c22469f90759ba53cedbdb5/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:21.306324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4cfda-5d43-4844-885d-66121299d412 None admin] RBAC: Authorizing `identity:create_grant(project_id=314e1f70a6da40a79963abc77a1351c4, group_id=a2395cbf5c22469f90759ba53cedbdb5, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:21.307831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4cfda-5d43-4844-885d-66121299d412 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:21.340361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 198/387] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Jun 29 08:22:21 2022] PUT /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4/groups/a2395cbf5c22469f90759ba53cedbdb5/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:21.488223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6208f8-0f70-4f78-b8d5-0bd3a4e94c57 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:21.488446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6208f8-0f70-4f78-b8d5-0bd3a4e94c57 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:21.488692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6208f8-0f70-4f78-b8d5-0bd3a4e94c57 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:21.490332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 190/388] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:21.496470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049bea3-757f-45c3-a826-2546013dc92b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:21.497095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049bea3-757f-45c3-a826-2546013dc92b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:21.497382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049bea3-757f-45c3-a826-2546013dc92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:21.506175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:21.522640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0049bea3-757f-45c3-a826-2546013dc92b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:21.523915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0049bea3-757f-45c3-a826-2546013dc92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:21.607496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 199/389] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:21.635223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6208f8-0f70-4f78-b8d5-0bd3a4e94c57 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:21.715692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:21.764612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6208f8-0f70-4f78-b8d5-0bd3a4e94c57 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:21.802313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110c5c5-b3f9-4d18-ae54-becc3c58e766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:21.803581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110c5c5-b3f9-4d18-ae54-becc3c58e766 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:21.804070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110c5c5-b3f9-4d18-ae54-becc3c58e766 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:21.804549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110c5c5-b3f9-4d18-ae54-becc3c58e766 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:21.816659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110c5c5-b3f9-4d18-ae54-becc3c58e766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5110c5c5-b3f9-4d18-ae54-becc3c58e766 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:21.825210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:21.826068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:21.826891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:21.834343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 191/390] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:22:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:21.835981 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049bea3-757f-45c3-a826-2546013dc92b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:21.953594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049bea3-757f-45c3-a826-2546013dc92b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:21.989891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:21.996127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf3939-08b2-456c-9789-a1766d5ffe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:21.997099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf3939-08b2-456c-9789-a1766d5ffe6c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:21.997582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf3939-08b2-456c-9789-a1766d5ffe6c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:21.998045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf3939-08b2-456c-9789-a1766d5ffe6c None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:21.999033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf3939-08b2-456c-9789-a1766d5ffe6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:21.999563 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf3939-08b2-456c-9789-a1766d5ffe6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:22.001130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf3939-08b2-456c-9789-a1766d5ffe6c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:22.014468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 200/391] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:22.020885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110c5c5-b3f9-4d18-ae54-becc3c58e766 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:22.144828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110c5c5-b3f9-4d18-ae54-becc3c58e766 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:22.163034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:22.181732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bc291-935f-4ea3-b500-ea7e1e74925a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:22.183010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bc291-935f-4ea3-b500-ea7e1e74925a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:22.183505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bc291-935f-4ea3-b500-ea7e1e74925a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:22.183968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bc291-935f-4ea3-b500-ea7e1e74925a None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:22.185370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60bc291-935f-4ea3-b500-ea7e1e74925a None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:22.187181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60bc291-935f-4ea3-b500-ea7e1e74925a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:22.193628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 192/392] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:22 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:22.195335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf3939-08b2-456c-9789-a1766d5ffe6c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:22.313647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf3939-08b2-456c-9789-a1766d5ffe6c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:22.356169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265114b9-3f4d-421a-b17f-7249a9390b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:22.357488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265114b9-3f4d-421a-b17f-7249a9390b72 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:22.357965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265114b9-3f4d-421a-b17f-7249a9390b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:22.358441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265114b9-3f4d-421a-b17f-7249a9390b72 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:22.360820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265114b9-3f4d-421a-b17f-7249a9390b72 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:22.362619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265114b9-3f4d-421a-b17f-7249a9390b72 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:22.364614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 201/393] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:22 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:22.370820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bc291-935f-4ea3-b500-ea7e1e74925a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:22.483507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bc291-935f-4ea3-b500-ea7e1e74925a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:22.521638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24114730-b509-4a1d-92a3-9fae762a2762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:22.522892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24114730-b509-4a1d-92a3-9fae762a2762 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:22.523367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24114730-b509-4a1d-92a3-9fae762a2762 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:22.523850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24114730-b509-4a1d-92a3-9fae762a2762 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4/groups/a2395cbf5c22469f90759ba53cedbdb5/roles/3531cd7306964cc5967f92c559890ae5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:22.526327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24114730-b509-4a1d-92a3-9fae762a2762 None admin] RBAC: Authorizing `identity:create_grant(project_id=314e1f70a6da40a79963abc77a1351c4, group_id=a2395cbf5c22469f90759ba53cedbdb5, role_id=3531cd7306964cc5967f92c559890ae5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:22.528169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24114730-b509-4a1d-92a3-9fae762a2762 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:22.552875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 193/394] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Jun 29 08:22:22 2022] PUT /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4/groups/a2395cbf5c22469f90759ba53cedbdb5/roles/3531cd7306964cc5967f92c559890ae5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:22.783860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20afecb-35fe-40bb-85af-3cbb15b09714 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:22.784249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20afecb-35fe-40bb-85af-3cbb15b09714 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:22.784582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20afecb-35fe-40bb-85af-3cbb15b09714 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:22.786385 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 202/395] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:22.792580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b8cd4-2b49-4d96-8dc7-454d67df8941 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:22.792874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b8cd4-2b49-4d96-8dc7-454d67df8941 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:22.793151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9b8cd4-2b49-4d96-8dc7-454d67df8941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:22.814197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da0321-b1c4-4b43-84a0-23fb8c273baf None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:22.814537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da0321-b1c4-4b43-84a0-23fb8c273baf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:22.814836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da0321-b1c4-4b43-84a0-23fb8c273baf None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:22.816497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 203/396] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:22:22.818753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8d9b8cd4-2b49-4d96-8dc7-454d67df8941 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:22.819960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8d9b8cd4-2b49-4d96-8dc7-454d67df8941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:22.822007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad79ef1-035e-48f1-9bb1-a5faae9ef989 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:22.822295 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad79ef1-035e-48f1-9bb1-a5faae9ef989 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:22.822575 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad79ef1-035e-48f1-9bb1-a5faae9ef989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:22.853749 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dad79ef1-035e-48f1-9bb1-a5faae9ef989 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:22.854989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dad79ef1-035e-48f1-9bb1-a5faae9ef989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:22.891958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:22.931338 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 194/397] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:22.940360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:22.954625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b8cd4-2b49-4d96-8dc7-454d67df8941 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:22.956098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 204/398] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:22.984072 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad79ef1-035e-48f1-9bb1-a5faae9ef989 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:23.117337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9b8cd4-2b49-4d96-8dc7-454d67df8941 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:23.158475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ec060-c9b3-47d1-ac5e-963c7f662270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:23.161113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ec060-c9b3-47d1-ac5e-963c7f662270 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:23.161399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ec060-c9b3-47d1-ac5e-963c7f662270 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:23.161695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ec060-c9b3-47d1-ac5e-963c7f662270 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:23.177201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad79ef1-035e-48f1-9bb1-a5faae9ef989 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:23.186493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ec060-c9b3-47d1-ac5e-963c7f662270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e45ec060-c9b3-47d1-ac5e-963c7f662270 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:23.194273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:23.195271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:23.196060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:23.200019 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 195/399] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:23.205486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ec060-c9b3-47d1-ac5e-963c7f662270 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:23.224088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1669f6-b65d-4b77-8c87-7816dfc795e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:23.224922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1669f6-b65d-4b77-8c87-7816dfc795e7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:23.225202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1669f6-b65d-4b77-8c87-7816dfc795e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:23.225473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1669f6-b65d-4b77-8c87-7816dfc795e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:23.236398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1669f6-b65d-4b77-8c87-7816dfc795e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5b1669f6-b65d-4b77-8c87-7816dfc795e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:23.249330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:23.250281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:23.251959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 205/400] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:22:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:23.256655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1669f6-b65d-4b77-8c87-7816dfc795e7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:23.257985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:23.317774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:23.342646 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ec060-c9b3-47d1-ac5e-963c7f662270 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:23.393491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1669f6-b65d-4b77-8c87-7816dfc795e7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:23.413525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b366a1-532a-48bd-bef6-1190b8772f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:23.414205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b366a1-532a-48bd-bef6-1190b8772f73 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:23.414483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b366a1-532a-48bd-bef6-1190b8772f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:23.414777 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b366a1-532a-48bd-bef6-1190b8772f73 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:23.415537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b366a1-532a-48bd-bef6-1190b8772f73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:23.415842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b366a1-532a-48bd-bef6-1190b8772f73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:23.417213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b366a1-532a-48bd-bef6-1190b8772f73 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:23.432249 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 196/401] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:23.433961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cb832-4a55-46a2-823a-5589b8a96030 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:23.434531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cb832-4a55-46a2-823a-5589b8a96030 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:23.435057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cb832-4a55-46a2-823a-5589b8a96030 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:23.436485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 197/402] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:23.439714 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cb832-4a55-46a2-823a-5589b8a96030 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:23.463167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41100d-e87e-4c1d-9c67-a2e690ecce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:23.463167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41100d-e87e-4c1d-9c67-a2e690ecce02 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:23.463167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41100d-e87e-4c1d-9c67-a2e690ecce02 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:23.463167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41100d-e87e-4c1d-9c67-a2e690ecce02 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:23.467230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41100d-e87e-4c1d-9c67-a2e690ecce02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:23.467230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41100d-e87e-4c1d-9c67-a2e690ecce02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:23.467230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41100d-e87e-4c1d-9c67-a2e690ecce02 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:23.478771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 206/403] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:23.480483 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc63e9-7017-418d-8b65-78cbbcc39e00 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:23.480970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc63e9-7017-418d-8b65-78cbbcc39e00 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:23.481432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc63e9-7017-418d-8b65-78cbbcc39e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:23.506832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dfcc63e9-7017-418d-8b65-78cbbcc39e00 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:23.508011 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dfcc63e9-7017-418d-8b65-78cbbcc39e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:23.569013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:23.577231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cb832-4a55-46a2-823a-5589b8a96030 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:23.613567 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dea35-d381-440e-98b2-f27d10a5eafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:23.613773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 207/404] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:23.614608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc63e9-7017-418d-8b65-78cbbcc39e00 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:23.615275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dea35-d381-440e-98b2-f27d10a5eafe None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:23.615755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dea35-d381-440e-98b2-f27d10a5eafe None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:23.616210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dea35-d381-440e-98b2-f27d10a5eafe None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:23.623642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dea35-d381-440e-98b2-f27d10a5eafe None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:23.625388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048dea35-d381-440e-98b2-f27d10a5eafe None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:23.631917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 198/405] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:23 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:23.635658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dea35-d381-440e-98b2-f27d10a5eafe None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:23.676089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:23.744808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc63e9-7017-418d-8b65-78cbbcc39e00 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:23.759764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048dea35-d381-440e-98b2-f27d10a5eafe None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:23.783198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e744f7-605a-40c1-969c-99c0fc9619c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:23.784519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e744f7-605a-40c1-969c-99c0fc9619c6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:23.784813 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e744f7-605a-40c1-969c-99c0fc9619c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:23.785092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e744f7-605a-40c1-969c-99c0fc9619c6 None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:23.786297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e744f7-605a-40c1-969c-99c0fc9619c6 None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:23.788029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e744f7-605a-40c1-969c-99c0fc9619c6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:23.794352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 208/406] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:23 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:23.795323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e744f7-605a-40c1-969c-99c0fc9619c6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:23.797532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad128ba2-083a-4c4a-8398-f16f9150eb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:23.798561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad128ba2-083a-4c4a-8398-f16f9150eb61 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:23.799025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad128ba2-083a-4c4a-8398-f16f9150eb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:23.799431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad128ba2-083a-4c4a-8398-f16f9150eb61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:23.810720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad128ba2-083a-4c4a-8398-f16f9150eb61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ad128ba2-083a-4c4a-8398-f16f9150eb61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:23.818711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:23.819757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:23.820665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:23.822971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 199/407] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:22:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:23.824012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad128ba2-083a-4c4a-8398-f16f9150eb61 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:23.851663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:23.908181 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e744f7-605a-40c1-969c-99c0fc9619c6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:23.923380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:23.929660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad128ba2-083a-4c4a-8398-f16f9150eb61 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:23.949051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ff2db-69df-46ee-8ac5-c37c68c8f342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:23.950154 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ff2db-69df-46ee-8ac5-c37c68c8f342 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:23.950599 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ff2db-69df-46ee-8ac5-c37c68c8f342 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:23.951137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ff2db-69df-46ee-8ac5-c37c68c8f342 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:23.952738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ff2db-69df-46ee-8ac5-c37c68c8f342 None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:23.955313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ff2db-69df-46ee-8ac5-c37c68c8f342 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:23.958084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 209/408] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:23 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:23.959202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ff2db-69df-46ee-8ac5-c37c68c8f342 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:23.966373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a263c3-6fbf-401f-8455-03383da9a401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:23.967329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a263c3-6fbf-401f-8455-03383da9a401 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:23.967610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a263c3-6fbf-401f-8455-03383da9a401 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:23.967884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a263c3-6fbf-401f-8455-03383da9a401 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:23.969132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a263c3-6fbf-401f-8455-03383da9a401 None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:23.970624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a263c3-6fbf-401f-8455-03383da9a401 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:23.977623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 200/409] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:23 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:23.977623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a263c3-6fbf-401f-8455-03383da9a401 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:24.070099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:24.078993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a263c3-6fbf-401f-8455-03383da9a401 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:24.085949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ff2db-69df-46ee-8ac5-c37c68c8f342 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:24.115031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5df89-653f-470c-8b5e-dfc4e2ea4419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:24.116139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5df89-653f-470c-8b5e-dfc4e2ea4419 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:24.116432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5df89-653f-470c-8b5e-dfc4e2ea4419 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:24.116774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a5df89-653f-470c-8b5e-dfc4e2ea4419 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c/groups/a2395cbf5c22469f90759ba53cedbdb5/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:24.118955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5df89-653f-470c-8b5e-dfc4e2ea4419 None admin] RBAC: Authorizing `identity:create_grant(project_id=c862664ec560416a936d940af366502c, group_id=a2395cbf5c22469f90759ba53cedbdb5, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:24.120493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a5df89-653f-470c-8b5e-dfc4e2ea4419 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:24.128079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8e70b-446b-4818-a3d6-6d18d64c8816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:24.129055 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8e70b-446b-4818-a3d6-6d18d64c8816 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:24.129565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8e70b-446b-4818-a3d6-6d18d64c8816 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:24.130009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8e70b-446b-4818-a3d6-6d18d64c8816 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:24.131140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8e70b-446b-4818-a3d6-6d18d64c8816 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:24.131742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8e70b-446b-4818-a3d6-6d18d64c8816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:24.133350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8e70b-446b-4818-a3d6-6d18d64c8816 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:24.136235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 201/410] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Jun 29 08:22:23 2022] PUT /identity/v3/projects/c862664ec560416a936d940af366502c/groups/a2395cbf5c22469f90759ba53cedbdb5/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:24.137194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5df89-653f-470c-8b5e-dfc4e2ea4419 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:24.149040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 210/411] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:24.151136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268ac68-4e42-41df-a71b-9af2a39a1b47 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:24.151731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268ac68-4e42-41df-a71b-9af2a39a1b47 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:24.152251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268ac68-4e42-41df-a71b-9af2a39a1b47 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:24.154113 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 211/412] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:24.155261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268ac68-4e42-41df-a71b-9af2a39a1b47 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:24.210805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:24.253663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a5df89-653f-470c-8b5e-dfc4e2ea4419 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:24.279127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268ac68-4e42-41df-a71b-9af2a39a1b47 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:24.292451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7afc1-86a9-4953-abb4-53506816dccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:24.293373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7afc1-86a9-4953-abb4-53506816dccb None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:24.293628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7afc1-86a9-4953-abb4-53506816dccb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:24.293897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7afc1-86a9-4953-abb4-53506816dccb None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c/groups/a2395cbf5c22469f90759ba53cedbdb5/roles/3531cd7306964cc5967f92c559890ae5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:24.296419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7afc1-86a9-4953-abb4-53506816dccb None admin] RBAC: Authorizing `identity:create_grant(project_id=c862664ec560416a936d940af366502c, group_id=a2395cbf5c22469f90759ba53cedbdb5, role_id=3531cd7306964cc5967f92c559890ae5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:24.297872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c7afc1-86a9-4953-abb4-53506816dccb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:24.316018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 202/413] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Jun 29 08:22:24 2022] PUT /identity/v3/projects/c862664ec560416a936d940af366502c/groups/a2395cbf5c22469f90759ba53cedbdb5/roles/3531cd7306964cc5967f92c559890ae5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:24.317609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bcd61-fe73-4833-983d-7e8d0028c2ab None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:24.317901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bcd61-fe73-4833-983d-7e8d0028c2ab None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:24.318325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82bcd61-fe73-4833-983d-7e8d0028c2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:24.320057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87c2fd-490f-4bb7-9839-35f823c56572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:24.321229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87c2fd-490f-4bb7-9839-35f823c56572 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:24.321760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87c2fd-490f-4bb7-9839-35f823c56572 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:24.322226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87c2fd-490f-4bb7-9839-35f823c56572 None admin] PATH_INFO: `/v3/groups/04745421c66d446499bbc4253685cb47` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:24.324114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87c2fd-490f-4bb7-9839-35f823c56572 None admin] RBAC: Authorizing `identity:get_group(group_id=04745421c66d446499bbc4253685cb47)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:24.327085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87c2fd-490f-4bb7-9839-35f823c56572 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:24.329695 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 212/414] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:24 2022] GET /identity/v3/groups/04745421c66d446499bbc4253685cb47 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:24.337471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87c2fd-490f-4bb7-9839-35f823c56572 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:24.342277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f82bcd61-fe73-4833-983d-7e8d0028c2ab None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:24.343511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f82bcd61-fe73-4833-983d-7e8d0028c2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:24.355056 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:24.478282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 203/415] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:24.501759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bcd61-fe73-4833-983d-7e8d0028c2ab None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:24.512339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87c2fd-490f-4bb7-9839-35f823c56572 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:24.524691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:24.549831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9a8e3-0925-4e88-92d6-3aea3bbedb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:24.550908 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9a8e3-0925-4e88-92d6-3aea3bbedb0c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:24.551191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9a8e3-0925-4e88-92d6-3aea3bbedb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:24.551463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9a8e3-0925-4e88-92d6-3aea3bbedb0c None admin] PATH_INFO: `/v3/projects/3b8a36f8c21e4c4f9394d7b2d795e615` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:24.552842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9a8e3-0925-4e88-92d6-3aea3bbedb0c None admin] RBAC: Authorizing `identity:get_project(project_id=3b8a36f8c21e4c4f9394d7b2d795e615)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:24.554472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9a8e3-0925-4e88-92d6-3aea3bbedb0c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:24.557221 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 213/416] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:24 2022] GET /identity/v3/projects/3b8a36f8c21e4c4f9394d7b2d795e615 => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:24.568970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9a8e3-0925-4e88-92d6-3aea3bbedb0c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:24.622145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82bcd61-fe73-4833-983d-7e8d0028c2ab None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:24.659113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf999f6-2ca9-45ce-b991-6445aae6361a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:24.660363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf999f6-2ca9-45ce-b991-6445aae6361a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:24.660869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf999f6-2ca9-45ce-b991-6445aae6361a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:24.661345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf999f6-2ca9-45ce-b991-6445aae6361a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:24.672660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf999f6-2ca9-45ce-b991-6445aae6361a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-caf999f6-2ca9-45ce-b991-6445aae6361a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:24.680882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:24.681762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:24.682521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:24.685381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 204/417] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:24.690840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf999f6-2ca9-45ce-b991-6445aae6361a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:24.694637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:24.721336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9a8e3-0925-4e88-92d6-3aea3bbedb0c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:24.791650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b21304-8e10-46e2-badb-946554aa40db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:24.792606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b21304-8e10-46e2-badb-946554aa40db None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:24.792950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b21304-8e10-46e2-badb-946554aa40db None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:24.793283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b21304-8e10-46e2-badb-946554aa40db None admin] PATH_INFO: `/v3/projects/3b8a36f8c21e4c4f9394d7b2d795e615/groups/04745421c66d446499bbc4253685cb47/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:24.798235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf999f6-2ca9-45ce-b991-6445aae6361a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:24.800218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b21304-8e10-46e2-badb-946554aa40db None admin] RBAC: Authorizing `identity:create_grant(project_id=3b8a36f8c21e4c4f9394d7b2d795e615, group_id=04745421c66d446499bbc4253685cb47, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:24.801798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b21304-8e10-46e2-badb-946554aa40db None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:24.833464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 214/418] 10.222.0.27 () {58 vars in 1442 bytes} [Wed Jun 29 08:22:24 2022] PUT /identity/v3/projects/3b8a36f8c21e4c4f9394d7b2d795e615/groups/04745421c66d446499bbc4253685cb47/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:24.835853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c38d8-cc19-4bce-a84f-f15c29c193b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:24.836836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c38d8-cc19-4bce-a84f-f15c29c193b5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:24.837395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c38d8-cc19-4bce-a84f-f15c29c193b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:24.837906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c38d8-cc19-4bce-a84f-f15c29c193b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:24.839299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c38d8-cc19-4bce-a84f-f15c29c193b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:24.839838 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c38d8-cc19-4bce-a84f-f15c29c193b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:24.841626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c38d8-cc19-4bce-a84f-f15c29c193b5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:24.853503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 205/419] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:24.860152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b21304-8e10-46e2-badb-946554aa40db None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:24.920487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:24.981320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1a595-1c34-4958-9140-d3ef83ac2c1b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:24.981665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1a595-1c34-4958-9140-d3ef83ac2c1b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:24.981868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb1a595-1c34-4958-9140-d3ef83ac2c1b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:24.985834 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 206/420] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:24.995721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b21304-8e10-46e2-badb-946554aa40db None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:25.000577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41100738-a893-4c9d-a746-71962fa73de4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:25.000892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41100738-a893-4c9d-a746-71962fa73de4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:25.001132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41100738-a893-4c9d-a746-71962fa73de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:25.035119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905dbb6-98c3-4f9d-a5ea-a843685f94b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:25.036239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905dbb6-98c3-4f9d-a5ea-a843685f94b2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:25.036728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905dbb6-98c3-4f9d-a5ea-a843685f94b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:25.037204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7905dbb6-98c3-4f9d-a5ea-a843685f94b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:25.049006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-41100738-a893-4c9d-a746-71962fa73de4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:25.050536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-41100738-a893-4c9d-a746-71962fa73de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:25.110826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905dbb6-98c3-4f9d-a5ea-a843685f94b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:25.112911 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7905dbb6-98c3-4f9d-a5ea-a843685f94b2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:25.146644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:25.154193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:25.156393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 207/421] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:25.194552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41100738-a893-4c9d-a746-71962fa73de4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:25.217768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 215/422] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:25.223686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905dbb6-98c3-4f9d-a5ea-a843685f94b2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:25.392059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41100738-a893-4c9d-a746-71962fa73de4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:25.396019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7905dbb6-98c3-4f9d-a5ea-a843685f94b2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:25.466723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b1c2d-65cd-466d-b32e-4e4108daba57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:25.466723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b1c2d-65cd-466d-b32e-4e4108daba57 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:25.466723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b1c2d-65cd-466d-b32e-4e4108daba57 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:25.466723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b1c2d-65cd-466d-b32e-4e4108daba57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:25.467021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ff5f2-a0bc-4f40-a91f-40fa6bd550dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:25.467575 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ff5f2-a0bc-4f40-a91f-40fa6bd550dd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:25.467856 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ff5f2-a0bc-4f40-a91f-40fa6bd550dd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:25.468129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ff5f2-a0bc-4f40-a91f-40fa6bd550dd None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:25.470710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ff5f2-a0bc-4f40-a91f-40fa6bd550dd None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:25.474564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ff5f2-a0bc-4f40-a91f-40fa6bd550dd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:25.482894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 216/423] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:25 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:25.488315 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ff5f2-a0bc-4f40-a91f-40fa6bd550dd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:25.499566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b1c2d-65cd-466d-b32e-4e4108daba57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:25.504166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-da5b1c2d-65cd-466d-b32e-4e4108daba57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:25.509903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:25.510912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:25.511699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 208/424] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:22:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:25.517197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b1c2d-65cd-466d-b32e-4e4108daba57 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:25.583115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:25.630387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b1c2d-65cd-466d-b32e-4e4108daba57 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:25.668116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e649c6-5d9e-4b7c-92c1-c35a5e61a410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:25.668933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e649c6-5d9e-4b7c-92c1-c35a5e61a410 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:25.669316 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e649c6-5d9e-4b7c-92c1-c35a5e61a410 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:25.669682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e649c6-5d9e-4b7c-92c1-c35a5e61a410 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:25.670827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e649c6-5d9e-4b7c-92c1-c35a5e61a410 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:25.671251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e649c6-5d9e-4b7c-92c1-c35a5e61a410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:25.673282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e649c6-5d9e-4b7c-92c1-c35a5e61a410 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:25.686023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 209/425] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:25.692172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e649c6-5d9e-4b7c-92c1-c35a5e61a410 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:25.781477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ff5f2-a0bc-4f40-a91f-40fa6bd550dd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:25.826684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e649c6-5d9e-4b7c-92c1-c35a5e61a410 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:25.845129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:25.866310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62060d09-5f88-485b-8283-aac131a6749c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:25.867268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060d09-5f88-485b-8283-aac131a6749c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:25.867701 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060d09-5f88-485b-8283-aac131a6749c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:25.868123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62060d09-5f88-485b-8283-aac131a6749c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:25.879417 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3a7f7-9584-461d-a42c-a5dab29927c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:25.880354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3a7f7-9584-461d-a42c-a5dab29927c6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:25.880575 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3a7f7-9584-461d-a42c-a5dab29927c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:25.880847 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3a7f7-9584-461d-a42c-a5dab29927c6 None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:25.887398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3a7f7-9584-461d-a42c-a5dab29927c6 None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:25.889077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3a7f7-9584-461d-a42c-a5dab29927c6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:25.890869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 217/426] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:25 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:25.896622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3a7f7-9584-461d-a42c-a5dab29927c6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:26.011881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:26.012484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62060d09-5f88-485b-8283-aac131a6749c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:26.013813 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62060d09-5f88-485b-8283-aac131a6749c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:26.026802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3a7f7-9584-461d-a42c-a5dab29927c6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:26.066849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92106-e984-402e-8a0d-536ae858f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:26.067508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92106-e984-402e-8a0d-536ae858f102 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:26.067789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92106-e984-402e-8a0d-536ae858f102 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:26.068062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee92106-e984-402e-8a0d-536ae858f102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:26.068891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee92106-e984-402e-8a0d-536ae858f102 None admin] RBAC: Adding query filter params (group.id=a2395cbf5c22469f90759ba53cedbdb5, role.id=b0d377da1a7d4260be1f7ae0715fda91, scope.project.id=314e1f70a6da40a79963abc77a1351c4) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:26.069200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee92106-e984-402e-8a0d-536ae858f102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:26.074710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee92106-e984-402e-8a0d-536ae858f102 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:26.076805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 218/427] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jun 29 08:22:25 2022] GET /identity/v3/role_assignments?group.id=a2395cbf5c22469f90759ba53cedbdb5&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:26.211444 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 210/428] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:26.217105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92106-e984-402e-8a0d-536ae858f102 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:26.347789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee92106-e984-402e-8a0d-536ae858f102 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:26.381908 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d2fb8-b6dc-40a2-9a4e-e51073dad1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:26.382806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d2fb8-b6dc-40a2-9a4e-e51073dad1f5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:26.383090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d2fb8-b6dc-40a2-9a4e-e51073dad1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:26.383362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d2fb8-b6dc-40a2-9a4e-e51073dad1f5 None admin] PATH_INFO: `/v3/projects/314e1f70a6da40a79963abc77a1351c4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:26.384562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d2fb8-b6dc-40a2-9a4e-e51073dad1f5 None admin] RBAC: Authorizing `identity:get_project(project_id=314e1f70a6da40a79963abc77a1351c4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:26.385995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d2fb8-b6dc-40a2-9a4e-e51073dad1f5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:26.388022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 219/429] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:26 2022] GET /identity/v3/projects/314e1f70a6da40a79963abc77a1351c4 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:26.393667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62060d09-5f88-485b-8283-aac131a6749c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:26.415130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:26.500107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583ef8-aa48-43fd-9100-f81f868e1254 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:26.500426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583ef8-aa48-43fd-9100-f81f868e1254 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:26.500646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78583ef8-aa48-43fd-9100-f81f868e1254 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:26.502260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 220/430] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:22:26.505313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62060d09-5f88-485b-8283-aac131a6749c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:26.515136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65474b2f-eda8-4468-8fcb-b438ec45c9a7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:26.515541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65474b2f-eda8-4468-8fcb-b438ec45c9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:26.515913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65474b2f-eda8-4468-8fcb-b438ec45c9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:26.532699 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:26.538605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-65474b2f-eda8-4468-8fcb-b438ec45c9a7 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:26.539517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-65474b2f-eda8-4468-8fcb-b438ec45c9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:26.561940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1383149-a587-456c-ba1f-509f7824dc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:26.563098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1383149-a587-456c-ba1f-509f7824dc40 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:26.563548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1383149-a587-456c-ba1f-509f7824dc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:26.563963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1383149-a587-456c-ba1f-509f7824dc40 None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:26.566683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1383149-a587-456c-ba1f-509f7824dc40 None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:26.568361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1383149-a587-456c-ba1f-509f7824dc40 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:26.570512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 211/431] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:26 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:26.576354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1383149-a587-456c-ba1f-509f7824dc40 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:26.631492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 221/432] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:26.657235 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65474b2f-eda8-4468-8fcb-b438ec45c9a7 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:26.684489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:26.709930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1383149-a587-456c-ba1f-509f7824dc40 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:26.763833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813fded-0c66-4582-97a7-669f8a0ee68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:26.764505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813fded-0c66-4582-97a7-669f8a0ee68d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:26.764783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813fded-0c66-4582-97a7-669f8a0ee68d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:26.765055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813fded-0c66-4582-97a7-669f8a0ee68d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:26.765901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813fded-0c66-4582-97a7-669f8a0ee68d None admin] RBAC: Adding query filter params (group.id=a2395cbf5c22469f90759ba53cedbdb5, role.id=3531cd7306964cc5967f92c559890ae5, scope.project.id=314e1f70a6da40a79963abc77a1351c4) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:26.766209 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813fded-0c66-4582-97a7-669f8a0ee68d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:26.769048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813fded-0c66-4582-97a7-669f8a0ee68d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:26.775885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 212/433] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jun 29 08:22:26 2022] GET /identity/v3/role_assignments?group.id=a2395cbf5c22469f90759ba53cedbdb5&scope.project.id=314e1f70a6da40a79963abc77a1351c4&role.id=3531cd7306964cc5967f92c559890ae5 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:26.794890 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65474b2f-eda8-4468-8fcb-b438ec45c9a7 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:26.834417 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad4592-be4d-4631-952e-09689041770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:26.835273 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad4592-be4d-4631-952e-09689041770e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:26.835553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad4592-be4d-4631-952e-09689041770e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:26.835826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ad4592-be4d-4631-952e-09689041770e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:26.839493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:26.846176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e116a-0770-44d5-9db6-6f558ccac9e2 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:26.847232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e116a-0770-44d5-9db6-6f558ccac9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:26.847733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e116a-0770-44d5-9db6-6f558ccac9e2 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:26.849537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 213/434] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:26.853710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ad4592-be4d-4631-952e-09689041770e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:26.857320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71f891-8bff-46f1-8121-ddbf9ae55459 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:26.857813 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71f891-8bff-46f1-8121-ddbf9ae55459 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a4ad4592-be4d-4631-952e-09689041770e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:26.858075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:26.858957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:26.861308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 222/435] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:26.862621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71f891-8bff-46f1-8121-ddbf9ae55459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:26.867823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad4592-be4d-4631-952e-09689041770e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:26.893823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ca71f891-8bff-46f1-8121-ddbf9ae55459 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:26.895429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ca71f891-8bff-46f1-8121-ddbf9ae55459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:26.993569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ad4592-be4d-4631-952e-09689041770e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:26.998450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 214/436] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:27.023703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71f891-8bff-46f1-8121-ddbf9ae55459 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:27.031507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba8daf-d9e7-4f6e-b890-9deb34215aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:27.032241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba8daf-d9e7-4f6e-b890-9deb34215aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:27.032538 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba8daf-d9e7-4f6e-b890-9deb34215aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:27.032820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba8daf-d9e7-4f6e-b890-9deb34215aa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:27.033595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba8daf-d9e7-4f6e-b890-9deb34215aa4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:27.035040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba8daf-d9e7-4f6e-b890-9deb34215aa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:27.035513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba8daf-d9e7-4f6e-b890-9deb34215aa4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:27.044955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 223/437] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:22:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:27.051447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba8daf-d9e7-4f6e-b890-9deb34215aa4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:27.124302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:27.131406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:27.159213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71f891-8bff-46f1-8121-ddbf9ae55459 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:27.186862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba8daf-d9e7-4f6e-b890-9deb34215aa4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:27.216319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256bff5-364f-437f-93b1-f9c562acd16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:27.217153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256bff5-364f-437f-93b1-f9c562acd16c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:27.217442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256bff5-364f-437f-93b1-f9c562acd16c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:27.217723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256bff5-364f-437f-93b1-f9c562acd16c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:27.231719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256bff5-364f-437f-93b1-f9c562acd16c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:27.235691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc272f-5965-4979-ae13-9cde455cb35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:27.236528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc272f-5965-4979-ae13-9cde455cb35f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:27.236961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc272f-5965-4979-ae13-9cde455cb35f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:27.237514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc272f-5965-4979-ae13-9cde455cb35f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d256bff5-364f-437f-93b1-f9c562acd16c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:27.240361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:27.241413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:27.242520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:27.246553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 215/438] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:22:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:27.247849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256bff5-364f-437f-93b1-f9c562acd16c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:27.331323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc272f-5965-4979-ae13-9cde455cb35f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:27.332782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc272f-5965-4979-ae13-9cde455cb35f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:27.364941 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256bff5-364f-437f-93b1-f9c562acd16c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:27.389227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:27.400211 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e38a41-af5e-4f1f-87e1-bac9974c0629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:27.400885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e38a41-af5e-4f1f-87e1-bac9974c0629 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:27.401168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e38a41-af5e-4f1f-87e1-bac9974c0629 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:27.401438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e38a41-af5e-4f1f-87e1-bac9974c0629 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:27.402193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e38a41-af5e-4f1f-87e1-bac9974c0629 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:27.402504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e38a41-af5e-4f1f-87e1-bac9974c0629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:27.403921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e38a41-af5e-4f1f-87e1-bac9974c0629 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:27.408985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:27.413785 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 216/439] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:27.415031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8eaaf-d2b7-4582-9970-7bcfebfae81e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:27.415497 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8eaaf-d2b7-4582-9970-7bcfebfae81e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:27.415772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8eaaf-d2b7-4582-9970-7bcfebfae81e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:27.417126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 217/440] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:27.419165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8eaaf-d2b7-4582-9970-7bcfebfae81e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:27.442057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 224/441] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:27.442057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008516d-7492-4c5b-b830-a1128b8446ae None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:27.442057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008516d-7492-4c5b-b830-a1128b8446ae None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:27.442057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008516d-7492-4c5b-b830-a1128b8446ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:27.455713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2008516d-7492-4c5b-b830-a1128b8446ae None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:27.456970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2008516d-7492-4c5b-b830-a1128b8446ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:27.522425 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 225/442] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:27.523294 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008516d-7492-4c5b-b830-a1128b8446ae None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:27.527814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8eaaf-d2b7-4582-9970-7bcfebfae81e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:27.564129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec8cb8-b7d1-4af6-8da5-6b8aa38a8674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:27.564790 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8cb8-b7d1-4af6-8da5-6b8aa38a8674 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:27.565075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8cb8-b7d1-4af6-8da5-6b8aa38a8674 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:27.565553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec8cb8-b7d1-4af6-8da5-6b8aa38a8674 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:27.610890 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:27.623205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008516d-7492-4c5b-b830-a1128b8446ae None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:27.634349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:27.636772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec8cb8-b7d1-4af6-8da5-6b8aa38a8674 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:27.638629 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec8cb8-b7d1-4af6-8da5-6b8aa38a8674 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:27.658492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b059743-a388-4bb9-b3b7-ea220fd89186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:27.659508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b059743-a388-4bb9-b3b7-ea220fd89186 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:27.659791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b059743-a388-4bb9-b3b7-ea220fd89186 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:27.660068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b059743-a388-4bb9-b3b7-ea220fd89186 None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:27.661320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b059743-a388-4bb9-b3b7-ea220fd89186 None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:27.662794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b059743-a388-4bb9-b3b7-ea220fd89186 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:27.664558 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 226/443] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:27 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:27.665264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b059743-a388-4bb9-b3b7-ea220fd89186 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:27.752421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 218/444] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 08:22:27.753609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec8cb8-b7d1-4af6-8da5-6b8aa38a8674 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:27.764520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b059743-a388-4bb9-b3b7-ea220fd89186 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:27.800700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589ba06-cefa-49db-874d-c5b1bfa671b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:27.801495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589ba06-cefa-49db-874d-c5b1bfa671b8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:27.801770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589ba06-cefa-49db-874d-c5b1bfa671b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:27.802040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589ba06-cefa-49db-874d-c5b1bfa671b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:27.812260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589ba06-cefa-49db-874d-c5b1bfa671b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3589ba06-cefa-49db-874d-c5b1bfa671b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:27.819778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:27.820646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:27.821519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 227/445] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:22:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:27.822162 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589ba06-cefa-49db-874d-c5b1bfa671b8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:27.840348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:27.857003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:27.860118 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec8cb8-b7d1-4af6-8da5-6b8aa38a8674 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:27.892404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9182826-c7e2-4e31-b3ee-25a98a87f37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:27.893457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9182826-c7e2-4e31-b3ee-25a98a87f37d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:27.893749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9182826-c7e2-4e31-b3ee-25a98a87f37d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:27.894020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9182826-c7e2-4e31-b3ee-25a98a87f37d None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:27.895209 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9182826-c7e2-4e31-b3ee-25a98a87f37d None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:27.896644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9182826-c7e2-4e31-b3ee-25a98a87f37d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:27.898261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 219/446] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:27 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:27.899031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9182826-c7e2-4e31-b3ee-25a98a87f37d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:27.926412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589ba06-cefa-49db-874d-c5b1bfa671b8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:27.959554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d687d-037c-417d-98c2-79a49522dcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:27.960442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d687d-037c-417d-98c2-79a49522dcbc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:27.960726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d687d-037c-417d-98c2-79a49522dcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:27.961001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d687d-037c-417d-98c2-79a49522dcbc None admin] PATH_INFO: `/v3/projects/c862664ec560416a936d940af366502c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:27.962200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d687d-037c-417d-98c2-79a49522dcbc None admin] RBAC: Authorizing `identity:get_project(project_id=c862664ec560416a936d940af366502c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:27.964100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d687d-037c-417d-98c2-79a49522dcbc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:27.965760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 228/447] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:27 2022] GET /identity/v3/projects/c862664ec560416a936d940af366502c => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:27.966499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d687d-037c-417d-98c2-79a49522dcbc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:28.003445 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9182826-c7e2-4e31-b3ee-25a98a87f37d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:28.039132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6d41f-629a-4ae4-b0d9-ac5fa11a62a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:28.039805 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6d41f-629a-4ae4-b0d9-ac5fa11a62a1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:28.040089 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6d41f-629a-4ae4-b0d9-ac5fa11a62a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:28.040365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6d41f-629a-4ae4-b0d9-ac5fa11a62a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:28.041116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6d41f-629a-4ae4-b0d9-ac5fa11a62a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:28.041431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6d41f-629a-4ae4-b0d9-ac5fa11a62a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:28.042829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6d41f-629a-4ae4-b0d9-ac5fa11a62a1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:28.051301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 220/448] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:28.052173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6d41f-629a-4ae4-b0d9-ac5fa11a62a1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:28.056771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:28.074170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d687d-037c-417d-98c2-79a49522dcbc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:28.082586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:28.110973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d73dbc-bbba-49cf-9886-5982aa941d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:28.111630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d73dbc-bbba-49cf-9886-5982aa941d91 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:28.111910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d73dbc-bbba-49cf-9886-5982aa941d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:28.112367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d73dbc-bbba-49cf-9886-5982aa941d91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:28.113443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d73dbc-bbba-49cf-9886-5982aa941d91 None admin] RBAC: Adding query filter params (group.id=a2395cbf5c22469f90759ba53cedbdb5, role.id=b0d377da1a7d4260be1f7ae0715fda91, scope.project.id=c862664ec560416a936d940af366502c) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:28.114048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d73dbc-bbba-49cf-9886-5982aa941d91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:28.115555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d73dbc-bbba-49cf-9886-5982aa941d91 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:28.121335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 229/449] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jun 29 08:22:27 2022] GET /identity/v3/role_assignments?group.id=a2395cbf5c22469f90759ba53cedbdb5&scope.project.id=c862664ec560416a936d940af366502c&role.id=b0d377da1a7d4260be1f7ae0715fda91 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:28.122078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d73dbc-bbba-49cf-9886-5982aa941d91 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:28.155552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6d41f-629a-4ae4-b0d9-ac5fa11a62a1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:28.193198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87701b67-3673-43c6-ba59-102b0c7fefea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:28.193997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87701b67-3673-43c6-ba59-102b0c7fefea None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:28.194280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87701b67-3673-43c6-ba59-102b0c7fefea None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:28.194558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87701b67-3673-43c6-ba59-102b0c7fefea None admin] PATH_INFO: `/v3/groups/a2395cbf5c22469f90759ba53cedbdb5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:28.195847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87701b67-3673-43c6-ba59-102b0c7fefea None admin] RBAC: Authorizing `identity:get_group(group_id=a2395cbf5c22469f90759ba53cedbdb5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:28.197336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87701b67-3673-43c6-ba59-102b0c7fefea None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:28.199069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 221/450] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:28 2022] GET /identity/v3/groups/a2395cbf5c22469f90759ba53cedbdb5 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:28.204939 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87701b67-3673-43c6-ba59-102b0c7fefea None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:28.260350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d73dbc-bbba-49cf-9886-5982aa941d91 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:28.298973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346f895-713d-4273-ac6f-951c30259680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:28.299746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346f895-713d-4273-ac6f-951c30259680 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:28.300690 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346f895-713d-4273-ac6f-951c30259680 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:28.300690 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346f895-713d-4273-ac6f-951c30259680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:28.310795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:28.316456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87701b67-3673-43c6-ba59-102b0c7fefea None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:28.322860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:28.362205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d8600-c48c-4c33-8626-e650411d526d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:28.363138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d8600-c48c-4c33-8626-e650411d526d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:28.363567 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d8600-c48c-4c33-8626-e650411d526d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:28.363842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d8600-c48c-4c33-8626-e650411d526d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:28.364691 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d8600-c48c-4c33-8626-e650411d526d None admin] RBAC: Adding query filter params (group.id=a2395cbf5c22469f90759ba53cedbdb5, role.id=3531cd7306964cc5967f92c559890ae5, scope.project.id=c862664ec560416a936d940af366502c) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:28.365002 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d8600-c48c-4c33-8626-e650411d526d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:28.366448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d8600-c48c-4c33-8626-e650411d526d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:28.372446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 222/451] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jun 29 08:22:28 2022] GET /identity/v3/role_assignments?group.id=a2395cbf5c22469f90759ba53cedbdb5&scope.project.id=c862664ec560416a936d940af366502c&role.id=3531cd7306964cc5967f92c559890ae5 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:28.383186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346f895-713d-4273-ac6f-951c30259680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:28.384697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6346f895-713d-4273-ac6f-951c30259680 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:28.503442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 230/452] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:28.508882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d8600-c48c-4c33-8626-e650411d526d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:28.615018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d8600-c48c-4c33-8626-e650411d526d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:28.649277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cdeea-07f3-4b2c-83a3-a06068863076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:28.650201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cdeea-07f3-4b2c-83a3-a06068863076 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:28.650484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cdeea-07f3-4b2c-83a3-a06068863076 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:28.650790 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cdeea-07f3-4b2c-83a3-a06068863076 None admin] PATH_INFO: `/v3/projects/3b8a36f8c21e4c4f9394d7b2d795e615` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:28.652007 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cdeea-07f3-4b2c-83a3-a06068863076 None admin] RBAC: Authorizing `identity:get_project(project_id=3b8a36f8c21e4c4f9394d7b2d795e615)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:28.653447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cdeea-07f3-4b2c-83a3-a06068863076 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:28.655159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 223/453] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:22:28 2022] GET /identity/v3/projects/3b8a36f8c21e4c4f9394d7b2d795e615 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:28.661210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346f895-713d-4273-ac6f-951c30259680 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:28.711311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:28.761892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346f895-713d-4273-ac6f-951c30259680 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:28.795270 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e6a91-e7a8-4bd7-b0ab-df59cf2ee806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:28.796123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e6a91-e7a8-4bd7-b0ab-df59cf2ee806 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:28.796410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e6a91-e7a8-4bd7-b0ab-df59cf2ee806 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:28.796683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e6a91-e7a8-4bd7-b0ab-df59cf2ee806 None admin] PATH_INFO: `/v3/groups/04745421c66d446499bbc4253685cb47` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:28.797891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e6a91-e7a8-4bd7-b0ab-df59cf2ee806 None admin] RBAC: Authorizing `identity:get_group(group_id=04745421c66d446499bbc4253685cb47)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:28.799354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e6a91-e7a8-4bd7-b0ab-df59cf2ee806 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:28.801072 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 231/454] 10.222.0.27 () {56 vars in 1180 bytes} [Wed Jun 29 08:22:28 2022] GET /identity/v3/groups/04745421c66d446499bbc4253685cb47 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:28.810388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cdeea-07f3-4b2c-83a3-a06068863076 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:28.851173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:28.931392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cdeea-07f3-4b2c-83a3-a06068863076 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:28.967500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faed3d2-3b2f-4f17-9be4-f10458650636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:28.968083 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faed3d2-3b2f-4f17-9be4-f10458650636 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:28.968365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faed3d2-3b2f-4f17-9be4-f10458650636 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:28.968641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faed3d2-3b2f-4f17-9be4-f10458650636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:28.969467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faed3d2-3b2f-4f17-9be4-f10458650636 None admin] RBAC: Adding query filter params (group.id=04745421c66d446499bbc4253685cb47, role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=3b8a36f8c21e4c4f9394d7b2d795e615) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:28.969780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faed3d2-3b2f-4f17-9be4-f10458650636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:28.971252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faed3d2-3b2f-4f17-9be4-f10458650636 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:28.977316 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 224/455] 10.222.0.27 () {56 vars in 1376 bytes} [Wed Jun 29 08:22:28 2022] GET /identity/v3/role_assignments?group.id=04745421c66d446499bbc4253685cb47&scope.project.id=3b8a36f8c21e4c4f9394d7b2d795e615&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:31.439140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13fb52-d82c-4c23-a8bf-bf4b813e1080 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:31.440259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13fb52-d82c-4c23-a8bf-bf4b813e1080 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:31.440752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13fb52-d82c-4c23-a8bf-bf4b813e1080 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:31.442587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 232/456] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:31.455303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902e4a3-4134-4218-be4c-df0c81da1c9c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:31.456291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902e4a3-4134-4218-be4c-df0c81da1c9c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:31.456764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9902e4a3-4134-4218-be4c-df0c81da1c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:31.484001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9902e4a3-4134-4218-be4c-df0c81da1c9c None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:31.485203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9902e4a3-4134-4218-be4c-df0c81da1c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:31.566108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 225/457] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:31.608466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13fb52-d82c-4c23-a8bf-bf4b813e1080 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:31.675573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e386807-47d5-4314-a20f-a58cd952e489 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:31.675857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e386807-47d5-4314-a20f-a58cd952e489 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:31.676292 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e386807-47d5-4314-a20f-a58cd952e489 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:31.678107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 226/458] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:31.682841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc64e0b-718c-4be7-9ade-683ede02fc62 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:31.683281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc64e0b-718c-4be7-9ade-683ede02fc62 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:31.683693 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc64e0b-718c-4be7-9ade-683ede02fc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:31.708675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0cc64e0b-718c-4be7-9ade-683ede02fc62 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:31.710093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0cc64e0b-718c-4be7-9ade-683ede02fc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:31.736252 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13fb52-d82c-4c23-a8bf-bf4b813e1080 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:31.737756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:31.781525 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:31.786462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 227/459] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:31.809782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0dd0-9d8f-42dd-b976-930d1cf16001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:31.810596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0dd0-9d8f-42dd-b976-930d1cf16001 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:31.810910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0dd0-9d8f-42dd-b976-930d1cf16001 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:31.811186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0dd0-9d8f-42dd-b976-930d1cf16001 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:31.825696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc0dd0-9d8f-42dd-b976-930d1cf16001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:31.827225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc0dd0-9d8f-42dd-b976-930d1cf16001 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:31.830389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc64e0b-718c-4be7-9ade-683ede02fc62 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-74bc0dd0-9d8f-42dd-b976-930d1cf16001 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:31.848488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:31.849405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:31.850241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:31.850241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:31.850241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:31.850241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:31.850241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 233/460] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:31.859973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0dd0-9d8f-42dd-b976-930d1cf16001 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:31.952538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc64e0b-718c-4be7-9ade-683ede02fc62 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:31.994924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7487843-4981-44a6-88d3-75cb254d40c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:31.996027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7487843-4981-44a6-88d3-75cb254d40c3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:31.996492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7487843-4981-44a6-88d3-75cb254d40c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:31.996959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7487843-4981-44a6-88d3-75cb254d40c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.010396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7487843-4981-44a6-88d3-75cb254d40c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:32.011611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0dd0-9d8f-42dd-b976-930d1cf16001 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:32.012433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7487843-4981-44a6-88d3-75cb254d40c3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b7487843-4981-44a6-88d3-75cb254d40c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:32.034645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:32.035635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:32.036467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:32.036467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:32.036467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:32.036467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:32.042828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 228/461] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:32.044337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629ff1e-3eb7-4038-b1d6-c4f30cfff301 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.044797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629ff1e-3eb7-4038-b1d6-c4f30cfff301 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.045302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629ff1e-3eb7-4038-b1d6-c4f30cfff301 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.046844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 229/462] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:32.048327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629ff1e-3eb7-4038-b1d6-c4f30cfff301 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:32.048434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9de1c1-6479-4968-91e8-744256da4e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:32.049047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9de1c1-6479-4968-91e8-744256da4e24 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.049444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9de1c1-6479-4968-91e8-744256da4e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.049884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9de1c1-6479-4968-91e8-744256da4e24 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.050750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9de1c1-6479-4968-91e8-744256da4e24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:32.051058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9de1c1-6479-4968-91e8-744256da4e24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:32.052793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9de1c1-6479-4968-91e8-744256da4e24 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:32.059430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:32.068592 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 234/463] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:32.069875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623540bc-fae4-41ef-bc32-fe7ebfc0564c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.070160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623540bc-fae4-41ef-bc32-fe7ebfc0564c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.070434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623540bc-fae4-41ef-bc32-fe7ebfc0564c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.098751 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-623540bc-fae4-41ef-bc32-fe7ebfc0564c None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:32.101703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-623540bc-fae4-41ef-bc32-fe7ebfc0564c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:32.132290 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:32.172953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629ff1e-3eb7-4038-b1d6-c4f30cfff301 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:32.179141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 235/464] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:32.180257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623540bc-fae4-41ef-bc32-fe7ebfc0564c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:32.215244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9631911-84c7-4eb0-9da2-724a87bed629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:32.216364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9631911-84c7-4eb0-9da2-724a87bed629 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.216937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9631911-84c7-4eb0-9da2-724a87bed629 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.217405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9631911-84c7-4eb0-9da2-724a87bed629 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.218531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9631911-84c7-4eb0-9da2-724a87bed629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:32.219061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9631911-84c7-4eb0-9da2-724a87bed629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:32.220785 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9631911-84c7-4eb0-9da2-724a87bed629 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:32.240340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 230/465] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:32.245904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9631911-84c7-4eb0-9da2-724a87bed629 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:32.304327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623540bc-fae4-41ef-bc32-fe7ebfc0564c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:32.321580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:32.350712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb26388-e142-4bc5-9be8-af75da2995fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:32.351393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26388-e142-4bc5-9be8-af75da2995fa None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.351674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26388-e142-4bc5-9be8-af75da2995fa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.351958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb26388-e142-4bc5-9be8-af75da2995fa None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.353063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb26388-e142-4bc5-9be8-af75da2995fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:32.355061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb26388-e142-4bc5-9be8-af75da2995fa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:32.362018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9631911-84c7-4eb0-9da2-724a87bed629 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:32.390790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 236/466] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:22:32 2022] POST /identity/v3/users => generated 246 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:22:32.392107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb26388-e142-4bc5-9be8-af75da2995fa None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:32.424449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd83d02-5580-411f-8293-f49b9e80aff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:32.425695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd83d02-5580-411f-8293-f49b9e80aff1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.426166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd83d02-5580-411f-8293-f49b9e80aff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.426628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd83d02-5580-411f-8293-f49b9e80aff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.430421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:32.445503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd83d02-5580-411f-8293-f49b9e80aff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:32.447407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd83d02-5580-411f-8293-f49b9e80aff1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-8dd83d02-5580-411f-8293-f49b9e80aff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:32.472053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:32.473541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:32.474420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:32.474420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:32.474420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:32.474420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:32.474420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 231/467] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:32.479827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2d0a8-f32d-4c54-ad91-1c7f3431475e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.480025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2d0a8-f32d-4c54-ad91-1c7f3431475e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.480246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2d0a8-f32d-4c54-ad91-1c7f3431475e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.485938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 232/468] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:32 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:32.486951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2d0a8-f32d-4c54-ad91-1c7f3431475e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:32.551737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb26388-e142-4bc5-9be8-af75da2995fa None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:32.593081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b7e8c-e6d9-4118-84cd-4dab44eb0ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:32.594233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b7e8c-e6d9-4118-84cd-4dab44eb0ffd None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.594578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b7e8c-e6d9-4118-84cd-4dab44eb0ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.594896 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b7e8c-e6d9-4118-84cd-4dab44eb0ffd None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.596428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b7e8c-e6d9-4118-84cd-4dab44eb0ffd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:32.597795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b7e8c-e6d9-4118-84cd-4dab44eb0ffd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:32.620085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2d0a8-f32d-4c54-ad91-1c7f3431475e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:32.623389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 237/469] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:22:32 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:22:32.625003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe06e0d-ffb6-42bb-af91-ea6f34047c01 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.625552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe06e0d-ffb6-42bb-af91-ea6f34047c01 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.626015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe06e0d-ffb6-42bb-af91-ea6f34047c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.642123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:32.654184 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2fe06e0d-ffb6-42bb-af91-ea6f34047c01 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:32.655910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2fe06e0d-ffb6-42bb-af91-ea6f34047c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:32.668207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44a721-f77d-4ab6-bd78-e14609f5eb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:32.669020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44a721-f77d-4ab6-bd78-e14609f5eb3f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.669367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44a721-f77d-4ab6-bd78-e14609f5eb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.669703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44a721-f77d-4ab6-bd78-e14609f5eb3f None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.670513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44a721-f77d-4ab6-bd78-e14609f5eb3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:32.671573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44a721-f77d-4ab6-bd78-e14609f5eb3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:32.673064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44a721-f77d-4ab6-bd78-e14609f5eb3f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:32.688160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 233/470] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:32.694744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44a721-f77d-4ab6-bd78-e14609f5eb3f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:32.734483 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:32.740475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 238/471] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:32.768381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe06e0d-ffb6-42bb-af91-ea6f34047c01 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:32.827846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44a721-f77d-4ab6-bd78-e14609f5eb3f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:32.881495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2953c35-4f55-428f-bc16-61d7cd190a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:32.882176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2953c35-4f55-428f-bc16-61d7cd190a77 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.882451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2953c35-4f55-428f-bc16-61d7cd190a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.882744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2953c35-4f55-428f-bc16-61d7cd190a77 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.883465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2953c35-4f55-428f-bc16-61d7cd190a77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:32.884920 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2953c35-4f55-428f-bc16-61d7cd190a77 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:32.907607 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe06e0d-ffb6-42bb-af91-ea6f34047c01 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:32.912317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 234/472] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:22:32 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:22:32.913774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe94f5-5dbb-4261-b7c5-678ce2890ad2 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.914113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe94f5-5dbb-4261-b7c5-678ce2890ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.914391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe94f5-5dbb-4261-b7c5-678ce2890ad2 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.916013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 235/473] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:32.921866 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83c342-bfaf-422c-b879-16b3cd8fa229 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.922173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83c342-bfaf-422c-b879-16b3cd8fa229 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.922422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83c342-bfaf-422c-b879-16b3cd8fa229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.946909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4c83c342-bfaf-422c-b879-16b3cd8fa229 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:32.948190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4c83c342-bfaf-422c-b879-16b3cd8fa229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:32.963960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff64179-1f5a-46e7-8045-360287e4e8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:32.964827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff64179-1f5a-46e7-8045-360287e4e8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:32.965190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff64179-1f5a-46e7-8045-360287e4e8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:32.965465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff64179-1f5a-46e7-8045-360287e4e8d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:32.981864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:32.988853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff64179-1f5a-46e7-8045-360287e4e8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:32.991109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff64179-1f5a-46e7-8045-360287e4e8d8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-cff64179-1f5a-46e7-8045-360287e4e8d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:33.023792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:33.024700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:33.025601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:33.025601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:33.025601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:33.025601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:33.025601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 239/474] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:33.035976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff64179-1f5a-46e7-8045-360287e4e8d8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:33.036548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 236/475] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:33.057997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83c342-bfaf-422c-b879-16b3cd8fa229 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:33.173301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83c342-bfaf-422c-b879-16b3cd8fa229 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:33.197839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:33.212830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98423a7c-7a00-4d97-b11f-3c0b6c238b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:33.213915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98423a7c-7a00-4d97-b11f-3c0b6c238b65 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:33.214199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98423a7c-7a00-4d97-b11f-3c0b6c238b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:33.214547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98423a7c-7a00-4d97-b11f-3c0b6c238b65 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:33.215936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98423a7c-7a00-4d97-b11f-3c0b6c238b65 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:33.217329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98423a7c-7a00-4d97-b11f-3c0b6c238b65 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:33.227473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.sql.core [None req-98423a7c-7a00-4d97-b11f-3c0b6c238b65 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 08:22:33.227473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 29 08:22:33.227473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [parameters: {'id': 'ecb9075051444ad1b84f80f245d25297', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 29 08:22:33.227473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80130) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-98423a7c-7a00-4d97-b11f-3c0b6c238b65 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application conn.query(q) Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application result.read() Jun 29 08:22:33.246630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return ref.to_dict() Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 08:22:33.247591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application next(self.gen) Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application yield resource Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application next(self.gen) Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise self.value Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application session.commit() Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 08:22:33.248476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self.session.flush() Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application compat.raise_( Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 08:22:33.249310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 08:22:33.250158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application conn.query(q) Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application result.read() Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 08:22:33.250999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [parameters: {'id': 'ecb9075051444ad1b84f80f245d25297', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:33.251765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 29 08:22:33.252725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:33.253714 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 237/476] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Jun 29 08:22:33 2022] POST /identity/v3/roles => generated 144 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:33.255239 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98423a7c-7a00-4d97-b11f-3c0b6c238b65 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:33.267204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff64179-1f5a-46e7-8045-360287e4e8d8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:33.285631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:33.307902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff5447-d0e3-4c3b-9ad5-55b4aceb74ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:33.308914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff5447-d0e3-4c3b-9ad5-55b4aceb74ac None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:33.310438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff5447-d0e3-4c3b-9ad5-55b4aceb74ac None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:33.311012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff5447-d0e3-4c3b-9ad5-55b4aceb74ac None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:33.312005 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff5447-d0e3-4c3b-9ad5-55b4aceb74ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:33.312565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff5447-d0e3-4c3b-9ad5-55b4aceb74ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:33.314311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff5447-d0e3-4c3b-9ad5-55b4aceb74ac None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:33.328224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 240/477] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:33.333382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff5447-d0e3-4c3b-9ad5-55b4aceb74ac None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:33.363753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98423a7c-7a00-4d97-b11f-3c0b6c238b65 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:33.411053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c68a8-371e-41d3-a0a4-109ccbb62b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:33.411878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c68a8-371e-41d3-a0a4-109ccbb62b5d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:33.412156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c68a8-371e-41d3-a0a4-109ccbb62b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:33.412425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c68a8-371e-41d3-a0a4-109ccbb62b5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:33.429426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c68a8-371e-41d3-a0a4-109ccbb62b5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ab4c68a8-371e-41d3-a0a4-109ccbb62b5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:33.439515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:33.440598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 08:22:33.441395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:33.445705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 238/478] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:22:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:33.447344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c68a8-371e-41d3-a0a4-109ccbb62b5d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:33.485588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff5447-d0e3-4c3b-9ad5-55b4aceb74ac None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:33.504023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:33.534319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad950ae-69a4-4046-a555-5c59f94cec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:33.535157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad950ae-69a4-4046-a555-5c59f94cec31 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:33.535335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad950ae-69a4-4046-a555-5c59f94cec31 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:33.535568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad950ae-69a4-4046-a555-5c59f94cec31 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:33.536340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad950ae-69a4-4046-a555-5c59f94cec31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:33.537722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad950ae-69a4-4046-a555-5c59f94cec31 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:33.569167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:33.575617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 241/479] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:22:33 2022] POST /identity/v3/users => generated 249 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:22:33.586460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c68a8-371e-41d3-a0a4-109ccbb62b5d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:33.625663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36b3fc-ae39-4b94-9939-4ee6dcbcf194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:33.626777 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36b3fc-ae39-4b94-9939-4ee6dcbcf194 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:33.627461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36b3fc-ae39-4b94-9939-4ee6dcbcf194 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:33.628097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36b3fc-ae39-4b94-9939-4ee6dcbcf194 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:33.629082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36b3fc-ae39-4b94-9939-4ee6dcbcf194 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:33.629592 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36b3fc-ae39-4b94-9939-4ee6dcbcf194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:33.631403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36b3fc-ae39-4b94-9939-4ee6dcbcf194 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:33.641832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 239/480] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:34.851986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6d60d-ea6e-4451-ab9f-71fdc614777b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:34.852499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6d60d-ea6e-4451-ab9f-71fdc614777b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:34.852627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6d60d-ea6e-4451-ab9f-71fdc614777b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:34.854155 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 242/481] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:34.859526 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2726f2d-47fe-4f52-bf30-68198616d812 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:34.860344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2726f2d-47fe-4f52-bf30-68198616d812 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:34.860815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2726f2d-47fe-4f52-bf30-68198616d812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:34.882329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c2726f2d-47fe-4f52-bf30-68198616d812 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:34.883845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c2726f2d-47fe-4f52-bf30-68198616d812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:34.973771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 240/482] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:35.010361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6d60d-ea6e-4451-ab9f-71fdc614777b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:35.172320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6d60d-ea6e-4451-ab9f-71fdc614777b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:35.213244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639688d-c92c-4187-add5-fe82b417a14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:35.214387 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639688d-c92c-4187-add5-fe82b417a14a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:35.214670 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639688d-c92c-4187-add5-fe82b417a14a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:35.214976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639688d-c92c-4187-add5-fe82b417a14a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:35.226171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639688d-c92c-4187-add5-fe82b417a14a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8639688d-c92c-4187-add5-fe82b417a14a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:35.233895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:35.234806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:35.235610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 243/483] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:35.243846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2726f2d-47fe-4f52-bf30-68198616d812 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:35.280502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a0cd-9060-472a-8073-3592179ab285 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:35.280846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a0cd-9060-472a-8073-3592179ab285 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:35.281057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5a0cd-9060-472a-8073-3592179ab285 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:35.282722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 244/484] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:22:35.287754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c990a-b54a-4d5a-abf3-772255b0a420 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:35.288041 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c990a-b54a-4d5a-abf3-772255b0a420 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:35.288314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c990a-b54a-4d5a-abf3-772255b0a420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:35.305627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3e8c990a-b54a-4d5a-abf3-772255b0a420 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:35.306835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3e8c990a-b54a-4d5a-abf3-772255b0a420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:35.330000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:35.379713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 245/485] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:35.415040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:35.431786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2726f2d-47fe-4f52-bf30-68198616d812 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:35.438127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c990a-b54a-4d5a-abf3-772255b0a420 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:35.512339 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc2d58-beeb-4dad-8161-786c76769e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:35.513031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc2d58-beeb-4dad-8161-786c76769e1b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:35.513324 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc2d58-beeb-4dad-8161-786c76769e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:35.513615 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc2d58-beeb-4dad-8161-786c76769e1b None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:35.514388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc2d58-beeb-4dad-8161-786c76769e1b None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:35.514817 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc2d58-beeb-4dad-8161-786c76769e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:35.516240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc2d58-beeb-4dad-8161-786c76769e1b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:35.525494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 241/486] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:35.530883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc2d58-beeb-4dad-8161-786c76769e1b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:35.631007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c990a-b54a-4d5a-abf3-772255b0a420 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:35.672367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc2d58-beeb-4dad-8161-786c76769e1b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:35.694276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170ea7d-d010-41fc-bb95-5d0ff97fe29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:35.695038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170ea7d-d010-41fc-bb95-5d0ff97fe29a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:35.695320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170ea7d-d010-41fc-bb95-5d0ff97fe29a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:35.695597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170ea7d-d010-41fc-bb95-5d0ff97fe29a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:35.717276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170ea7d-d010-41fc-bb95-5d0ff97fe29a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0170ea7d-d010-41fc-bb95-5d0ff97fe29a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:35.725620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:35.726504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:35.727333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:35.736579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 246/487] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:35.740858 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db3201-d976-4012-9f3f-a2e6f810dc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:35.742629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084263ee-c72b-4b4c-aa59-a8c5bef7a177 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:35.742950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084263ee-c72b-4b4c-aa59-a8c5bef7a177 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:35.743228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084263ee-c72b-4b4c-aa59-a8c5bef7a177 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:35.744043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db3201-d976-4012-9f3f-a2e6f810dc32 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:35.744532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db3201-d976-4012-9f3f-a2e6f810dc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:35.744829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 247/488] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:22:35.745190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db3201-d976-4012-9f3f-a2e6f810dc32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:35.745525 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084263ee-c72b-4b4c-aa59-a8c5bef7a177 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:35.760753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db3201-d976-4012-9f3f-a2e6f810dc32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:35.762535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db3201-d976-4012-9f3f-a2e6f810dc32 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:35.764701 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:35.769138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-24db3201-d976-4012-9f3f-a2e6f810dc32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:35.785756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:35.786666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:35.787534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:35.787534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:35.787534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:35.787534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:35.790791 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 242/489] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:35.792187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aadc99-f10a-47af-a609-16db39d05140 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:35.792477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aadc99-f10a-47af-a609-16db39d05140 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:35.792758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aadc99-f10a-47af-a609-16db39d05140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:35.811961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-81aadc99-f10a-47af-a609-16db39d05140 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:35.813315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-81aadc99-f10a-47af-a609-16db39d05140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:35.875972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084263ee-c72b-4b4c-aa59-a8c5bef7a177 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:35.894785 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 243/490] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:35.897112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aadc99-f10a-47af-a609-16db39d05140 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:35.940163 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c9656-f004-4b64-9dce-547062e8ea49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:35.941049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c9656-f004-4b64-9dce-547062e8ea49 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:35.941504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c9656-f004-4b64-9dce-547062e8ea49 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:35.941847 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c9656-f004-4b64-9dce-547062e8ea49 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:35.942951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c9656-f004-4b64-9dce-547062e8ea49 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:35.943334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c9656-f004-4b64-9dce-547062e8ea49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:35.945545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c9656-f004-4b64-9dce-547062e8ea49 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:35.957341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 248/491] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:35.958318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c9656-f004-4b64-9dce-547062e8ea49 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:36.024523 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aadc99-f10a-47af-a609-16db39d05140 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:36.051873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:36.068368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:36.071807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f0231-8e8d-4c45-b67e-02b2c22a9222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:36.072707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f0231-8e8d-4c45-b67e-02b2c22a9222 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.073352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f0231-8e8d-4c45-b67e-02b2c22a9222 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.073802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f0231-8e8d-4c45-b67e-02b2c22a9222 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.075084 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f0231-8e8d-4c45-b67e-02b2c22a9222 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:36.075570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f0231-8e8d-4c45-b67e-02b2c22a9222 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:36.077233 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f0231-8e8d-4c45-b67e-02b2c22a9222 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:36.091898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c9656-f004-4b64-9dce-547062e8ea49 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:36.094807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 244/492] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:36.100232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f0231-8e8d-4c45-b67e-02b2c22a9222 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:36.135803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c2f35-c538-4105-843b-fe233347f198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:36.136609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2f35-c538-4105-843b-fe233347f198 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.136891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2f35-c538-4105-843b-fe233347f198 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.137167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2f35-c538-4105-843b-fe233347f198 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.147884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c2f35-c538-4105-843b-fe233347f198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f90c2f35-c538-4105-843b-fe233347f198 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:36.155604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:36.156602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:36.157392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 249/493] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:36.158035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c2f35-c538-4105-843b-fe233347f198 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:36.222967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f0231-8e8d-4c45-b67e-02b2c22a9222 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:36.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a2a2a-2432-492e-a9f5-bbbcd29d048b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:36.257498 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a2a2a-2432-492e-a9f5-bbbcd29d048b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.257826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a2a2a-2432-492e-a9f5-bbbcd29d048b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.258097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a2a2a-2432-492e-a9f5-bbbcd29d048b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.266850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c2f35-c538-4105-843b-fe233347f198 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:36.270310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a2a2a-2432-492e-a9f5-bbbcd29d048b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:36.271807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a2a2a-2432-492e-a9f5-bbbcd29d048b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d40a2a2a-2432-492e-a9f5-bbbcd29d048b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:36.290376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:36.291284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:36.292105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:36.292105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:36.292105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:36.292105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:36.292105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 245/494] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:36.292950 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e155-4a68-4cac-945e-242a589fdebb None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.293234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e155-4a68-4cac-945e-242a589fdebb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.293506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1e155-4a68-4cac-945e-242a589fdebb None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.299040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 246/495] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:36.299863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1e155-4a68-4cac-945e-242a589fdebb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:36.306909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec57a33-50f8-4220-adc3-5812ab226061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:36.307709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec57a33-50f8-4220-adc3-5812ab226061 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.307992 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec57a33-50f8-4220-adc3-5812ab226061 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.308263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec57a33-50f8-4220-adc3-5812ab226061 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.316069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec57a33-50f8-4220-adc3-5812ab226061 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:36.317527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec57a33-50f8-4220-adc3-5812ab226061 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:36.320445 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-eec57a33-50f8-4220-adc3-5812ab226061 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:36.333559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:36.334641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 250/496] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jun 29 08:22:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:36.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec57a33-50f8-4220-adc3-5812ab226061 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:36.342581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:36.413358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1e155-4a68-4cac-945e-242a589fdebb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:36.435066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec57a33-50f8-4220-adc3-5812ab226061 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:36.448000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe32272-2e42-49cd-a011-0559f885227f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:36.448665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe32272-2e42-49cd-a011-0559f885227f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.448948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe32272-2e42-49cd-a011-0559f885227f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.449218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe32272-2e42-49cd-a011-0559f885227f None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.449957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe32272-2e42-49cd-a011-0559f885227f None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:36.450262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe32272-2e42-49cd-a011-0559f885227f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:36.451649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe32272-2e42-49cd-a011-0559f885227f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:36.462180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 247/497] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:36.463901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d4d24-96c1-4df1-b0ca-5b70bc7d08a0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.464192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d4d24-96c1-4df1-b0ca-5b70bc7d08a0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.464475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d4d24-96c1-4df1-b0ca-5b70bc7d08a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.471853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca2310-0307-45e1-aa21-41314de3022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:36.472509 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca2310-0307-45e1-aa21-41314de3022a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.472791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca2310-0307-45e1-aa21-41314de3022a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.473126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca2310-0307-45e1-aa21-41314de3022a None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.473786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca2310-0307-45e1-aa21-41314de3022a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:36.474087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca2310-0307-45e1-aa21-41314de3022a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:36.475489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca2310-0307-45e1-aa21-41314de3022a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:36.481761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ce7d4d24-96c1-4df1-b0ca-5b70bc7d08a0 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:36.482977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ce7d4d24-96c1-4df1-b0ca-5b70bc7d08a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:36.487418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 251/498] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:36.488647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0d16-7d45-4ffa-a730-2d7bce0e73b3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.488928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0d16-7d45-4ffa-a730-2d7bce0e73b3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.489198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c0d16-7d45-4ffa-a730-2d7bce0e73b3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.490558 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 252/499] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:36.491105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c0d16-7d45-4ffa-a730-2d7bce0e73b3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:36.552480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:36.554143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 248/500] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:36.555112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d4d24-96c1-4df1-b0ca-5b70bc7d08a0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:36.583710 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:36.594742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c0d16-7d45-4ffa-a730-2d7bce0e73b3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:36.628933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0cb46-ea59-452b-a9ba-acbb4e304cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:36.629605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0cb46-ea59-452b-a9ba-acbb4e304cbf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.629888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0cb46-ea59-452b-a9ba-acbb4e304cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.630169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0cb46-ea59-452b-a9ba-acbb4e304cbf None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.631047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0cb46-ea59-452b-a9ba-acbb4e304cbf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:36.631368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0cb46-ea59-452b-a9ba-acbb4e304cbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:36.632829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0cb46-ea59-452b-a9ba-acbb4e304cbf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:36.654559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d4d24-96c1-4df1-b0ca-5b70bc7d08a0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:36.669506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 253/501] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jun 29 08:22:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:36.670335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0cb46-ea59-452b-a9ba-acbb4e304cbf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:36.689787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ac7ef-879c-4a5a-a0a9-c1a87b38d3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:36.690756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ac7ef-879c-4a5a-a0a9-c1a87b38d3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.691058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ac7ef-879c-4a5a-a0a9-c1a87b38d3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.691333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ac7ef-879c-4a5a-a0a9-c1a87b38d3e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.701815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ac7ef-879c-4a5a-a0a9-c1a87b38d3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:36.703376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ac7ef-879c-4a5a-a0a9-c1a87b38d3e3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-702ac7ef-879c-4a5a-a0a9-c1a87b38d3e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:36.720894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:36.722103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:36.722979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:36.722979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:36.722979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:36.722979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:36.722979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 249/502] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:36.723498 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ebaa0-7a52-406f-b8c9-48598b795e75 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.723694 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ebaa0-7a52-406f-b8c9-48598b795e75 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.723996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ebaa0-7a52-406f-b8c9-48598b795e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.740087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-013ebaa0-7a52-406f-b8c9-48598b795e75 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:36.741258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-013ebaa0-7a52-406f-b8c9-48598b795e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:36.771255 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0cb46-ea59-452b-a9ba-acbb4e304cbf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:36.788386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:36.806156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a960558-c493-4226-b3a5-f89b9a521e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:36.806994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a960558-c493-4226-b3a5-f89b9a521e8e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.807277 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a960558-c493-4226-b3a5-f89b9a521e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.807568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a960558-c493-4226-b3a5-f89b9a521e8e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.807994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 250/503] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:36.808826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ebaa0-7a52-406f-b8c9-48598b795e75 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:36.820482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:36.827228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a960558-c493-4226-b3a5-f89b9a521e8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:36.829042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a960558-c493-4226-b3a5-f89b9a521e8e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7a960558-c493-4226-b3a5-f89b9a521e8e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:36.844620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:36.845522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:36.846418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:36.846418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:36.846418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:36.846418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:36.846418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:36.846418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:36.846418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:36.846418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:36.846418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 08:22:36.846418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:36.846418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 254/504] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:36.847687 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a960558-c493-4226-b3a5-f89b9a521e8e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:36.915562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ebaa0-7a52-406f-b8c9-48598b795e75 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:36.948017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979f7aa-3c71-4de1-8f74-ce62f40c33a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:36.948815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979f7aa-3c71-4de1-8f74-ce62f40c33a8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.949097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979f7aa-3c71-4de1-8f74-ce62f40c33a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.949369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979f7aa-3c71-4de1-8f74-ce62f40c33a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:36.955171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a960558-c493-4226-b3a5-f89b9a521e8e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:36.959849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979f7aa-3c71-4de1-8f74-ce62f40c33a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:36.961316 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979f7aa-3c71-4de1-8f74-ce62f40c33a8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f979f7aa-3c71-4de1-8f74-ce62f40c33a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:36.978904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:36.979747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:36.980598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:36.980598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:36.980598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:36.980598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:36.980598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 251/505] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:36.980870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979f7aa-3c71-4de1-8f74-ce62f40c33a8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:36.994090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5ce12-9955-4ea5-9df0-9e23d12491dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:36.994766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5ce12-9955-4ea5-9df0-9e23d12491dd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:36.995043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5ce12-9955-4ea5-9df0-9e23d12491dd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:36.995318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5ce12-9955-4ea5-9df0-9e23d12491dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:37.043617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:37.044399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:37.066944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5ce12-9955-4ea5-9df0-9e23d12491dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:37.068471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5ce12-9955-4ea5-9df0-9e23d12491dd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:37.083325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979f7aa-3c71-4de1-8f74-ce62f40c33a8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:37.118162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c7beb-e943-4cc1-8170-d68322944c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:37.118859 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c7beb-e943-4cc1-8170-d68322944c1b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:37.119140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c7beb-e943-4cc1-8170-d68322944c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:37.119413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c7beb-e943-4cc1-8170-d68322944c1b None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:37.120149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c7beb-e943-4cc1-8170-d68322944c1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:37.120473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c7beb-e943-4cc1-8170-d68322944c1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:37.121856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c7beb-e943-4cc1-8170-d68322944c1b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:37.133603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 252/506] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:37.135515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c7beb-e943-4cc1-8170-d68322944c1b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:37.169919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 255/507] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:37.170806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5ce12-9955-4ea5-9df0-9e23d12491dd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:37.239397 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c7beb-e943-4cc1-8170-d68322944c1b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:37.246703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:37.260975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:37.268580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5ce12-9955-4ea5-9df0-9e23d12491dd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:37.273512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847488ed-a0b1-41c4-b602-2bb4143ca987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:37.274311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847488ed-a0b1-41c4-b602-2bb4143ca987 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:37.274592 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847488ed-a0b1-41c4-b602-2bb4143ca987 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:37.274896 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847488ed-a0b1-41c4-b602-2bb4143ca987 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:37.285168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847488ed-a0b1-41c4-b602-2bb4143ca987 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-847488ed-a0b1-41c4-b602-2bb4143ca987 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:37.292786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:37.293695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:37.294482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 253/508] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:37.295157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847488ed-a0b1-41c4-b602-2bb4143ca987 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:37.307243 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3d8a3-611d-45cf-bf42-cd300dbeed18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:37.307991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3d8a3-611d-45cf-bf42-cd300dbeed18 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:37.308353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3d8a3-611d-45cf-bf42-cd300dbeed18 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:37.308633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3d8a3-611d-45cf-bf42-cd300dbeed18 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:37.309497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3d8a3-611d-45cf-bf42-cd300dbeed18 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:37.309811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3d8a3-611d-45cf-bf42-cd300dbeed18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:37.311290 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3d8a3-611d-45cf-bf42-cd300dbeed18 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:37.328556 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 256/509] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jun 29 08:22:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:37.329337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3d8a3-611d-45cf-bf42-cd300dbeed18 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:37.395003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847488ed-a0b1-41c4-b602-2bb4143ca987 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:37.427299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830cf7f-c52e-4ac2-b3c4-a941665bedce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:37.427961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830cf7f-c52e-4ac2-b3c4-a941665bedce None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:37.428245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830cf7f-c52e-4ac2-b3c4-a941665bedce None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:37.428335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3d8a3-611d-45cf-bf42-cd300dbeed18 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:37.428523 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830cf7f-c52e-4ac2-b3c4-a941665bedce None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:37.429250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830cf7f-c52e-4ac2-b3c4-a941665bedce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:37.429566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830cf7f-c52e-4ac2-b3c4-a941665bedce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:37.430973 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830cf7f-c52e-4ac2-b3c4-a941665bedce None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:37.442590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 254/510] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:37.443488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830cf7f-c52e-4ac2-b3c4-a941665bedce None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:37.471162 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89d2a2-d7c2-481b-b4a9-1dea25163eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:37.472137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89d2a2-d7c2-481b-b4a9-1dea25163eec None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:37.472405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89d2a2-d7c2-481b-b4a9-1dea25163eec None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:37.472696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89d2a2-d7c2-481b-b4a9-1dea25163eec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:37.475662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:37.480998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89d2a2-d7c2-481b-b4a9-1dea25163eec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:37.482478 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89d2a2-d7c2-481b-b4a9-1dea25163eec None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:37.488473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ff89d2a2-d7c2-481b-b4a9-1dea25163eec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:37.496969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:37.498017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 257/511] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:37.499438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89d2a2-d7c2-481b-b4a9-1dea25163eec None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:37.546818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830cf7f-c52e-4ac2-b3c4-a941665bedce None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:37.580123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e1a35-1bda-458a-ba63-368268dbd2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:37.580998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1a35-1bda-458a-ba63-368268dbd2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:37.581276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1a35-1bda-458a-ba63-368268dbd2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:37.581543 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1a35-1bda-458a-ba63-368268dbd2f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:37.591798 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e1a35-1bda-458a-ba63-368268dbd2f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:37.593288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e1a35-1bda-458a-ba63-368268dbd2f8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:37.599131 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89d2a2-d7c2-481b-b4a9-1dea25163eec None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-532e1a35-1bda-458a-ba63-368268dbd2f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:37.610337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:37.611302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:37.612181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:37.612181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:37.612181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:37.612181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:37.612181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 255/512] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:37.612447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e1a35-1bda-458a-ba63-368268dbd2f8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:37.634917 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faf29e-1129-48eb-9a3d-4fce9101de12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:37.635551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faf29e-1129-48eb-9a3d-4fce9101de12 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:37.635832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faf29e-1129-48eb-9a3d-4fce9101de12 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:37.636107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faf29e-1129-48eb-9a3d-4fce9101de12 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:37.636853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23faf29e-1129-48eb-9a3d-4fce9101de12 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:37.637160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23faf29e-1129-48eb-9a3d-4fce9101de12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:37.638533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23faf29e-1129-48eb-9a3d-4fce9101de12 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:37.646886 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 258/513] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:37.647468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faf29e-1129-48eb-9a3d-4fce9101de12 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:37.703589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:37.709302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:37.716511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e1a35-1bda-458a-ba63-368268dbd2f8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:37.746622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faf29e-1129-48eb-9a3d-4fce9101de12 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:37.750024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d86e9f-29c3-4efa-ae07-07c0d9d4eb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:37.750619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d86e9f-29c3-4efa-ae07-07c0d9d4eb5b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:37.750927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d86e9f-29c3-4efa-ae07-07c0d9d4eb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:37.751197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d86e9f-29c3-4efa-ae07-07c0d9d4eb5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:37.778972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65e79d-206f-4d11-ae8d-d14ff1fc309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:37.779629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e79d-206f-4d11-ae8d-d14ff1fc309a None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:37.779908 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e79d-206f-4d11-ae8d-d14ff1fc309a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:37.780181 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65e79d-206f-4d11-ae8d-d14ff1fc309a None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:37.780897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65e79d-206f-4d11-ae8d-d14ff1fc309a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:37.782357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65e79d-206f-4d11-ae8d-d14ff1fc309a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:37.802867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 259/514] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:22:37 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:22:37.803849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65e79d-206f-4d11-ae8d-d14ff1fc309a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:37.833438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d86e9f-29c3-4efa-ae07-07c0d9d4eb5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:37.834994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d86e9f-29c3-4efa-ae07-07c0d9d4eb5b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:37.921666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65e79d-206f-4d11-ae8d-d14ff1fc309a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:37.937224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:37.949417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:37.959451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 256/515] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:37.959949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d86e9f-29c3-4efa-ae07-07c0d9d4eb5b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:37.962163 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50885c02-7891-4bbd-8391-1b9417673ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:37.962876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50885c02-7891-4bbd-8391-1b9417673ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:37.963166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50885c02-7891-4bbd-8391-1b9417673ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:37.963792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50885c02-7891-4bbd-8391-1b9417673ed6 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:37.964650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50885c02-7891-4bbd-8391-1b9417673ed6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:37.964966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50885c02-7891-4bbd-8391-1b9417673ed6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:37.966524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50885c02-7891-4bbd-8391-1b9417673ed6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:37.986428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 260/516] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jun 29 08:22:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:37.988131 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50885c02-7891-4bbd-8391-1b9417673ed6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:38.073905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d86e9f-29c3-4efa-ae07-07c0d9d4eb5b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:38.096317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50885c02-7891-4bbd-8391-1b9417673ed6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:38.108685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff517a-845b-493f-b47d-f420cbb1577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:38.109373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff517a-845b-493f-b47d-f420cbb1577d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:38.109649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff517a-845b-493f-b47d-f420cbb1577d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:38.109925 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff517a-845b-493f-b47d-f420cbb1577d None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:38.110654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff517a-845b-493f-b47d-f420cbb1577d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:38.110984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff517a-845b-493f-b47d-f420cbb1577d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:38.112700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff517a-845b-493f-b47d-f420cbb1577d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:38.123992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 257/517] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:38.124797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff517a-845b-493f-b47d-f420cbb1577d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:38.136257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee579e8-2a67-465b-8866-07478d8fe7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:38.137057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee579e8-2a67-465b-8866-07478d8fe7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:38.137342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee579e8-2a67-465b-8866-07478d8fe7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:38.137624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee579e8-2a67-465b-8866-07478d8fe7b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:38.147709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee579e8-2a67-465b-8866-07478d8fe7b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:38.149196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee579e8-2a67-465b-8866-07478d8fe7b1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5ee579e8-2a67-465b-8866-07478d8fe7b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:38.168263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:38.169142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:38.169985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:38.169985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:38.169985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:38.169985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:38.169985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 261/518] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:38.170678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee579e8-2a67-465b-8866-07478d8fe7b1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:38.188934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:38.202866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:38.234037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff517a-845b-493f-b47d-f420cbb1577d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:38.267329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489af9b7-da50-49d5-914c-9fccbcc9ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:38.268235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af9b7-da50-49d5-914c-9fccbcc9ccf7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:38.268644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af9b7-da50-49d5-914c-9fccbcc9ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:38.268924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af9b7-da50-49d5-914c-9fccbcc9ccf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:38.279281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489af9b7-da50-49d5-914c-9fccbcc9ccf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:38.280732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489af9b7-da50-49d5-914c-9fccbcc9ccf7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:38.282664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee579e8-2a67-465b-8866-07478d8fe7b1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-489af9b7-da50-49d5-914c-9fccbcc9ccf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:38.297539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:38.298604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:38.299755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:38.299755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:38.299755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:38.299755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:38.303416 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 258/519] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:38.304285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489af9b7-da50-49d5-914c-9fccbcc9ccf7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:38.325040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed12afc-a145-43d7-8939-117305e481b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:38.325962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed12afc-a145-43d7-8939-117305e481b0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:38.326539 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed12afc-a145-43d7-8939-117305e481b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:38.326874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed12afc-a145-43d7-8939-117305e481b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:38.398202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed12afc-a145-43d7-8939-117305e481b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:38.399719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed12afc-a145-43d7-8939-117305e481b0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:38.406881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489af9b7-da50-49d5-914c-9fccbcc9ccf7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:38.415105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:38.425355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:38.444693 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bd268-d0bb-432f-af3c-af374bb885e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:38.445491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bd268-d0bb-432f-af3c-af374bb885e0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:38.445768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bd268-d0bb-432f-af3c-af374bb885e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:38.446036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847bd268-d0bb-432f-af3c-af374bb885e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:38.459064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bd268-d0bb-432f-af3c-af374bb885e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:38.460605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847bd268-d0bb-432f-af3c-af374bb885e0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-847bd268-d0bb-432f-af3c-af374bb885e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:38.476169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:38.477040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:38.477867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:38.477867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:38.477867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:38.477867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:38.477867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:38.477867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:38.477867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:38.477867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:38.477867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 259/520] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:38.478331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bd268-d0bb-432f-af3c-af374bb885e0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:38.511908 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 262/521] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:38.511908 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed12afc-a145-43d7-8939-117305e481b0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:38.589360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847bd268-d0bb-432f-af3c-af374bb885e0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:38.610924 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed12afc-a145-43d7-8939-117305e481b0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:38.623681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce18802-4a6d-4cfd-a303-0728167dd131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:38.624342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce18802-4a6d-4cfd-a303-0728167dd131 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:38.624641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce18802-4a6d-4cfd-a303-0728167dd131 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:38.624880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce18802-4a6d-4cfd-a303-0728167dd131 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:38.625708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce18802-4a6d-4cfd-a303-0728167dd131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:38.625918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce18802-4a6d-4cfd-a303-0728167dd131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:38.627310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce18802-4a6d-4cfd-a303-0728167dd131 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:38.637026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:38.640401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 260/522] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:38.641196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce18802-4a6d-4cfd-a303-0728167dd131 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:38.652896 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9eaef-ed53-471e-b9a4-e6ea8ae13f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:38.653561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9eaef-ed53-471e-b9a4-e6ea8ae13f69 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:38.653845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9eaef-ed53-471e-b9a4-e6ea8ae13f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:38.654201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9eaef-ed53-471e-b9a4-e6ea8ae13f69 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:38.654878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9eaef-ed53-471e-b9a4-e6ea8ae13f69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:38.655193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9eaef-ed53-471e-b9a4-e6ea8ae13f69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:38.656581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9eaef-ed53-471e-b9a4-e6ea8ae13f69 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:38.668412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 263/523] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:38.669199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:38.673761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9eaef-ed53-471e-b9a4-e6ea8ae13f69 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:38.744551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce18802-4a6d-4cfd-a303-0728167dd131 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:38.776554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9eaef-ed53-471e-b9a4-e6ea8ae13f69 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:38.779772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2ad9d-4ba8-4d55-afbe-c13644686262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:38.780462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2ad9d-4ba8-4d55-afbe-c13644686262 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:38.780741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2ad9d-4ba8-4d55-afbe-c13644686262 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:38.781013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2ad9d-4ba8-4d55-afbe-c13644686262 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:38.781799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2ad9d-4ba8-4d55-afbe-c13644686262 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:38.782105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2ad9d-4ba8-4d55-afbe-c13644686262 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:38.783702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2ad9d-4ba8-4d55-afbe-c13644686262 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:38.802304 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 261/524] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:38.803258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2ad9d-4ba8-4d55-afbe-c13644686262 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:38.816357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94278e3-2a71-4c0a-b461-f4eb0673aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:38.817308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94278e3-2a71-4c0a-b461-f4eb0673aef3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:38.817589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94278e3-2a71-4c0a-b461-f4eb0673aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:38.817862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94278e3-2a71-4c0a-b461-f4eb0673aef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:38.829926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94278e3-2a71-4c0a-b461-f4eb0673aef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:38.831436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94278e3-2a71-4c0a-b461-f4eb0673aef3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d94278e3-2a71-4c0a-b461-f4eb0673aef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:38.850332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:38.851372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:38.852231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:38.852231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:38.852231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:38.852231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:38.852231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 264/525] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:38.852231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94278e3-2a71-4c0a-b461-f4eb0673aef3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:38.887583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:38.896623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:38.912668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2ad9d-4ba8-4d55-afbe-c13644686262 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:38.950342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c22f3e-80f4-49a7-aa37-9899d040ac77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:38.951188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c22f3e-80f4-49a7-aa37-9899d040ac77 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:38.951473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c22f3e-80f4-49a7-aa37-9899d040ac77 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:38.951741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c22f3e-80f4-49a7-aa37-9899d040ac77 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:38.961407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c22f3e-80f4-49a7-aa37-9899d040ac77 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:38.962916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c22f3e-80f4-49a7-aa37-9899d040ac77 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:38.965764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94278e3-2a71-4c0a-b461-f4eb0673aef3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a4c22f3e-80f4-49a7-aa37-9899d040ac77 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:38.979169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:38.980070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:38.981041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:38.981041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:38.981041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:38.981041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:38.981041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:38.981041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:38.981041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:38.981041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:38.981041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 08:22:38.981041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:38.984914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 262/526] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:38.985706 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c22f3e-80f4-49a7-aa37-9899d040ac77 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:39.003393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f457a1-65ce-4512-9dca-d0b42d1c49d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:39.004440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f457a1-65ce-4512-9dca-d0b42d1c49d0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:39.004721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f457a1-65ce-4512-9dca-d0b42d1c49d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:39.005142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f457a1-65ce-4512-9dca-d0b42d1c49d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:39.013007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f457a1-65ce-4512-9dca-d0b42d1c49d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:39.014460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f457a1-65ce-4512-9dca-d0b42d1c49d0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e9f457a1-65ce-4512-9dca-d0b42d1c49d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:39.029564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:39.030449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:39.031345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:39.031345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:39.031345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:39.031345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:39.031345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:39.031345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:39.031345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:39.031345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:39.031345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 265/527] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:39.031875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f457a1-65ce-4512-9dca-d0b42d1c49d0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:39.094364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c22f3e-80f4-49a7-aa37-9899d040ac77 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:39.132208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a69f7-f935-430e-9c42-4290a8708baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:39.132784 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a69f7-f935-430e-9c42-4290a8708baf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:39.133079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a69f7-f935-430e-9c42-4290a8708baf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:39.133330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a69f7-f935-430e-9c42-4290a8708baf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:39.134162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a69f7-f935-430e-9c42-4290a8708baf None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=7bd500f020f143339c5a4fc9f0728ee4) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:39.134472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a69f7-f935-430e-9c42-4290a8708baf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:39.135971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a69f7-f935-430e-9c42-4290a8708baf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:39.137814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:39.150719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 263/528] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:38 2022] GET /identity/v3/role_assignments?user.id=7bd500f020f143339c5a4fc9f0728ee4&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:39.150719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a69f7-f935-430e-9c42-4290a8708baf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:39.154559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f457a1-65ce-4512-9dca-d0b42d1c49d0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:39.163022 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:39.202449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a000c-9eff-4377-849a-7cfc5334955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:39.203150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a000c-9eff-4377-849a-7cfc5334955b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:39.203684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a000c-9eff-4377-849a-7cfc5334955b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:39.203962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a000c-9eff-4377-849a-7cfc5334955b None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:39.205097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a000c-9eff-4377-849a-7cfc5334955b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:39.205406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a000c-9eff-4377-849a-7cfc5334955b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:39.206963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a000c-9eff-4377-849a-7cfc5334955b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:39.223667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 266/529] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:39.224647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a000c-9eff-4377-849a-7cfc5334955b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:39.271174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a69f7-f935-430e-9c42-4290a8708baf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:39.309851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97e033-a44d-49c3-9cd1-dc45a879bb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:39.310538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97e033-a44d-49c3-9cd1-dc45a879bb90 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:39.310867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97e033-a44d-49c3-9cd1-dc45a879bb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:39.311141 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97e033-a44d-49c3-9cd1-dc45a879bb90 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:39.312018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97e033-a44d-49c3-9cd1-dc45a879bb90 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:39.312402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97e033-a44d-49c3-9cd1-dc45a879bb90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:39.314031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97e033-a44d-49c3-9cd1-dc45a879bb90 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:39.355237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a000c-9eff-4377-849a-7cfc5334955b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:39.355641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 264/530] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Jun 29 08:22:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:39.356561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97e033-a44d-49c3-9cd1-dc45a879bb90 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:39.403843 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384904a-eaf5-4ddf-a482-9eb144e9dab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:39.404696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384904a-eaf5-4ddf-a482-9eb144e9dab6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:39.404983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384904a-eaf5-4ddf-a482-9eb144e9dab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:39.405261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384904a-eaf5-4ddf-a482-9eb144e9dab6 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:39.406316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384904a-eaf5-4ddf-a482-9eb144e9dab6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:39.406626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384904a-eaf5-4ddf-a482-9eb144e9dab6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:39.408912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384904a-eaf5-4ddf-a482-9eb144e9dab6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:39.428971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 267/531] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:39.429962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384904a-eaf5-4ddf-a482-9eb144e9dab6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:39.436265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:39.459504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:39.484560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97e033-a44d-49c3-9cd1-dc45a879bb90 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:39.528493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e6fb8-355c-46e9-808d-b4a41764c2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:39.529396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e6fb8-355c-46e9-808d-b4a41764c2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:39.529755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e6fb8-355c-46e9-808d-b4a41764c2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:39.530271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e6fb8-355c-46e9-808d-b4a41764c2c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:39.538215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e6fb8-355c-46e9-808d-b4a41764c2c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:39.539882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e6fb8-355c-46e9-808d-b4a41764c2c8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e79e6fb8-355c-46e9-808d-b4a41764c2c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:39.555115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:39.556009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:39.556832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:39.556832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:39.556832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:39.556832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:39.556832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:39.556832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:39.556832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:39.556832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:39.561257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 265/532] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:39.562070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384904a-eaf5-4ddf-a482-9eb144e9dab6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:39.562181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e6fb8-355c-46e9-808d-b4a41764c2c8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:39.601033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db069ff-a5e1-450b-b905-c61366715a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:39.601769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db069ff-a5e1-450b-b905-c61366715a4c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:39.602114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db069ff-a5e1-450b-b905-c61366715a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:39.602650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db069ff-a5e1-450b-b905-c61366715a4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:39.680533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db069ff-a5e1-450b-b905-c61366715a4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:39.683047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db069ff-a5e1-450b-b905-c61366715a4c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:39.687536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e6fb8-355c-46e9-808d-b4a41764c2c8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:39.705790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:39.716215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:39.725477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424b576-f65a-4ea8-8aaa-6bcf252ce910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:39.726410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424b576-f65a-4ea8-8aaa-6bcf252ce910 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:39.726754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424b576-f65a-4ea8-8aaa-6bcf252ce910 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:39.727120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424b576-f65a-4ea8-8aaa-6bcf252ce910 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:39.728055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424b576-f65a-4ea8-8aaa-6bcf252ce910 None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=fd09453b5db645d7b03522f91effb8d4) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:39.728476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424b576-f65a-4ea8-8aaa-6bcf252ce910 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:39.730177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424b576-f65a-4ea8-8aaa-6bcf252ce910 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:39.743138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 266/533] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:39 2022] GET /identity/v3/role_assignments?user.id=fd09453b5db645d7b03522f91effb8d4&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:39.744353 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424b576-f65a-4ea8-8aaa-6bcf252ce910 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:39.807460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 268/534] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:39.816034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db069ff-a5e1-450b-b905-c61366715a4c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:39.942534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424b576-f65a-4ea8-8aaa-6bcf252ce910 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:39.945874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db069ff-a5e1-450b-b905-c61366715a4c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:39.967137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:39.981479 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc8640-c88a-4997-947e-c766b14bd4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:39.982856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc8640-c88a-4997-947e-c766b14bd4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:39.983356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc8640-c88a-4997-947e-c766b14bd4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:39.983815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc8640-c88a-4997-947e-c766b14bd4d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:39.984793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc8640-c88a-4997-947e-c766b14bd4d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:39.985296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc8640-c88a-4997-947e-c766b14bd4d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:39.987434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc8640-c88a-4997-947e-c766b14bd4d3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:39.988809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae324fba-2ab4-4042-9386-9a3267583a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:39.989773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae324fba-2ab4-4042-9386-9a3267583a83 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:39.990059 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae324fba-2ab4-4042-9386-9a3267583a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:39.990328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae324fba-2ab4-4042-9386-9a3267583a83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:40.001094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 267/535] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:40.002639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc4357-60c4-4555-8206-45d9c96306e5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:40.003163 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc4357-60c4-4555-8206-45d9c96306e5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:40.003626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc4357-60c4-4555-8206-45d9c96306e5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:40.005296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 268/536] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:40.007221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc4357-60c4-4555-8206-45d9c96306e5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:40.007518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae324fba-2ab4-4042-9386-9a3267583a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:40.009048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae324fba-2ab4-4042-9386-9a3267583a83 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ae324fba-2ab4-4042-9386-9a3267583a83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:40.031450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:40.032342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:40.033150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:40.033150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:40.033150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:40.033150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:40.034759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 269/537] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:40.036206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180226-6d35-42e5-885a-1c4c8cd2ff59 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:40.036670 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180226-6d35-42e5-885a-1c4c8cd2ff59 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:40.037160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca180226-6d35-42e5-885a-1c4c8cd2ff59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:40.058049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ca180226-6d35-42e5-885a-1c4c8cd2ff59 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:40.061024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ca180226-6d35-42e5-885a-1c4c8cd2ff59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:40.078816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:40.131471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 270/538] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:40.132723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca180226-6d35-42e5-885a-1c4c8cd2ff59 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:40.155130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc4357-60c4-4555-8206-45d9c96306e5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:40.196342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca7d72-6b79-4b22-9421-4c1ea6126f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:40.196937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca7d72-6b79-4b22-9421-4c1ea6126f31 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:40.197221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca7d72-6b79-4b22-9421-4c1ea6126f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:40.197490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca7d72-6b79-4b22-9421-4c1ea6126f31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:40.198322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca7d72-6b79-4b22-9421-4c1ea6126f31 None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=b60ca33edf5a47fdab6e39cc98638c6d) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:40.198705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca7d72-6b79-4b22-9421-4c1ea6126f31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:40.200087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca7d72-6b79-4b22-9421-4c1ea6126f31 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:40.215756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 269/539] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:40 2022] GET /identity/v3/role_assignments?user.id=b60ca33edf5a47fdab6e39cc98638c6d&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:40.216156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca7d72-6b79-4b22-9421-4c1ea6126f31 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:40.245833 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:40.250915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca180226-6d35-42e5-885a-1c4c8cd2ff59 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:40.302466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70004c66-ab1d-4484-9a67-3d84258e2483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:40.303706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70004c66-ab1d-4484-9a67-3d84258e2483 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:40.305114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70004c66-ab1d-4484-9a67-3d84258e2483 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:40.305754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70004c66-ab1d-4484-9a67-3d84258e2483 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:40.307853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70004c66-ab1d-4484-9a67-3d84258e2483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:40.308587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70004c66-ab1d-4484-9a67-3d84258e2483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:40.311733 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70004c66-ab1d-4484-9a67-3d84258e2483 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:40.339488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 271/540] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:40.347845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70004c66-ab1d-4484-9a67-3d84258e2483 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:40.434051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca7d72-6b79-4b22-9421-4c1ea6126f31 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:40.458144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:40.477215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136f2d1-0c92-4445-b126-b8bb194a0b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:40.478164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136f2d1-0c92-4445-b126-b8bb194a0b50 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:40.478391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136f2d1-0c92-4445-b126-b8bb194a0b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:40.478667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136f2d1-0c92-4445-b126-b8bb194a0b50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:40.485172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70004c66-ab1d-4484-9a67-3d84258e2483 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:40.490369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136f2d1-0c92-4445-b126-b8bb194a0b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-9136f2d1-0c92-4445-b126-b8bb194a0b50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:40.498391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:40.499269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:40.500048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 270/541] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:40.505514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136f2d1-0c92-4445-b126-b8bb194a0b50 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:40.525500 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a9c35-71ee-43b1-a2f8-a08459f84947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:40.526673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a9c35-71ee-43b1-a2f8-a08459f84947 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:40.527182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a9c35-71ee-43b1-a2f8-a08459f84947 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:40.527639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a9c35-71ee-43b1-a2f8-a08459f84947 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:40.535123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a9c35-71ee-43b1-a2f8-a08459f84947 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:40.536853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a9c35-71ee-43b1-a2f8-a08459f84947 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-296a9c35-71ee-43b1-a2f8-a08459f84947 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:40.551368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:40.552214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:40.553063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:40.553063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:40.553063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:40.553063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:40.553063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:40.553063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:40.553063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:40.553063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:40.555610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 272/542] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:40.570169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a9c35-71ee-43b1-a2f8-a08459f84947 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:40.603077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:40.635672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136f2d1-0c92-4445-b126-b8bb194a0b50 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:40.669519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e33cde-c307-4f07-93b9-b15359f682be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:40.670234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e33cde-c307-4f07-93b9-b15359f682be None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:40.670518 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e33cde-c307-4f07-93b9-b15359f682be None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:40.670821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e33cde-c307-4f07-93b9-b15359f682be None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:40.671635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e33cde-c307-4f07-93b9-b15359f682be None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:40.671943 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e33cde-c307-4f07-93b9-b15359f682be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:40.673315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e33cde-c307-4f07-93b9-b15359f682be None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:40.682253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 271/543] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:40.687662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e33cde-c307-4f07-93b9-b15359f682be None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:40.716654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a9c35-71ee-43b1-a2f8-a08459f84947 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:40.769264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:40.785976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab7627-130b-44c7-802d-1fb346fbb7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:40.786711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab7627-130b-44c7-802d-1fb346fbb7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:40.786993 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab7627-130b-44c7-802d-1fb346fbb7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:40.787265 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab7627-130b-44c7-802d-1fb346fbb7b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:40.788073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab7627-130b-44c7-802d-1fb346fbb7b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:40.788381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab7627-130b-44c7-802d-1fb346fbb7b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:40.790261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab7627-130b-44c7-802d-1fb346fbb7b5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:40.812051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 273/544] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:40.820280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab7627-130b-44c7-802d-1fb346fbb7b5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:40.880796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e33cde-c307-4f07-93b9-b15359f682be None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:40.915165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb99ed-be12-4ff9-8320-c949d7043dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:40.916369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb99ed-be12-4ff9-8320-c949d7043dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:40.916841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb99ed-be12-4ff9-8320-c949d7043dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:40.917319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb99ed-be12-4ff9-8320-c949d7043dc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:40.928032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb99ed-be12-4ff9-8320-c949d7043dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:40.929759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb99ed-be12-4ff9-8320-c949d7043dc4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d2eb99ed-be12-4ff9-8320-c949d7043dc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:40.947742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:40.948605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:40.949424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:40.949424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:40.949424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:40.949424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:40.951631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 272/545] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:40.957355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb99ed-be12-4ff9-8320-c949d7043dc4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:40.989161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:41.048726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab7627-130b-44c7-802d-1fb346fbb7b5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:41.066991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:41.083468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb99ed-be12-4ff9-8320-c949d7043dc4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:41.107378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079e2f7-9cfd-454a-84e6-ff085ff67243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:41.108040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079e2f7-9cfd-454a-84e6-ff085ff67243 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:41.108320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079e2f7-9cfd-454a-84e6-ff085ff67243 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:41.108593 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079e2f7-9cfd-454a-84e6-ff085ff67243 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:41.109450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079e2f7-9cfd-454a-84e6-ff085ff67243 None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=49a1439029fa4b719bc114c3f59e0050) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:41.109760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079e2f7-9cfd-454a-84e6-ff085ff67243 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:41.111229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079e2f7-9cfd-454a-84e6-ff085ff67243 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:41.121734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 274/546] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:40 2022] GET /identity/v3/role_assignments?user.id=49a1439029fa4b719bc114c3f59e0050&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:41.149155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6898be-eba6-454d-b2e1-6ec60a21d957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:41.149853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6898be-eba6-454d-b2e1-6ec60a21d957 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:41.150133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6898be-eba6-454d-b2e1-6ec60a21d957 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:41.150407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6898be-eba6-454d-b2e1-6ec60a21d957 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:41.151480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6898be-eba6-454d-b2e1-6ec60a21d957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:41.151843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6898be-eba6-454d-b2e1-6ec60a21d957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:41.153377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6898be-eba6-454d-b2e1-6ec60a21d957 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:41.167755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 273/547] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:41.173481 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079e2f7-9cfd-454a-84e6-ff085ff67243 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:41.326832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079e2f7-9cfd-454a-84e6-ff085ff67243 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:41.377287 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0c6f6-57cf-4bc1-904d-0f8b043c22cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:41.378150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0c6f6-57cf-4bc1-904d-0f8b043c22cb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:41.378436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0c6f6-57cf-4bc1-904d-0f8b043c22cb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:41.378780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0c6f6-57cf-4bc1-904d-0f8b043c22cb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:41.384225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbd65d-8648-41d1-8079-4e751234e559 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:41.384512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbd65d-8648-41d1-8079-4e751234e559 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:41.384784 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbd65d-8648-41d1-8079-4e751234e559 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:41.386382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 274/548] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:41.388199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0c6f6-57cf-4bc1-904d-0f8b043c22cb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:41.389662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0c6f6-57cf-4bc1-904d-0f8b043c22cb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:41.391468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000df21-71df-4429-b341-5e7ba7732d80 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:41.391825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000df21-71df-4429-b341-5e7ba7732d80 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:41.392108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000df21-71df-4429-b341-5e7ba7732d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-70c0c6f6-57cf-4bc1-904d-0f8b043c22cb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:41.411710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:41.412650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:41.415044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:41.415044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:41.415044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:41.415044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:41.415044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:41.415044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:41.415044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:41.415044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:41.415044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:22:41.415044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:41.415044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 275/549] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:22:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:41.418761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0c6f6-57cf-4bc1-904d-0f8b043c22cb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:41.427008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:41.427252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b000df21-71df-4429-b341-5e7ba7732d80 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:41.428486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b000df21-71df-4429-b341-5e7ba7732d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:41.566735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 275/550] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:41.590110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0c6f6-57cf-4bc1-904d-0f8b043c22cb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:41.600807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000df21-71df-4429-b341-5e7ba7732d80 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:41.655771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e27144-c388-4acb-8f2b-b964c9518669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:41.656276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e27144-c388-4acb-8f2b-b964c9518669 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:41.656562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e27144-c388-4acb-8f2b-b964c9518669 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:41.656834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e27144-c388-4acb-8f2b-b964c9518669 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:41.657691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e27144-c388-4acb-8f2b-b964c9518669 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:41.658003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e27144-c388-4acb-8f2b-b964c9518669 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:41.659659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e27144-c388-4acb-8f2b-b964c9518669 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:41.687501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 276/551] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jun 29 08:22:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:41.696281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e27144-c388-4acb-8f2b-b964c9518669 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:41.770964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:41.789829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:41.807760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000df21-71df-4429-b341-5e7ba7732d80 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:41.860995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e27144-c388-4acb-8f2b-b964c9518669 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:41.887963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef03a06-d06d-4bfd-81b4-24af872b83d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:41.889079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03a06-d06d-4bfd-81b4-24af872b83d7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:41.889560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03a06-d06d-4bfd-81b4-24af872b83d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:41.890004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef03a06-d06d-4bfd-81b4-24af872b83d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:41.906277 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ee44b-2f86-4524-911f-0a567122626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:41.906953 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ee44b-2f86-4524-911f-0a567122626c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:41.907236 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ee44b-2f86-4524-911f-0a567122626c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:41.907511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ee44b-2f86-4524-911f-0a567122626c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:41.918575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef03a06-d06d-4bfd-81b4-24af872b83d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3ef03a06-d06d-4bfd-81b4-24af872b83d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:41.929886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:41.931505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:41.932907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:41.937274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 276/552] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:41.942184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef03a06-d06d-4bfd-81b4-24af872b83d7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:41.997031 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ee44b-2f86-4524-911f-0a567122626c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:41.998514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ee44b-2f86-4524-911f-0a567122626c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:42.059154 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:42.105118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 277/553] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:42.106586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299d53-4420-4394-a1e1-4e58e2d016c0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.106903 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299d53-4420-4394-a1e1-4e58e2d016c0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.107185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299d53-4420-4394-a1e1-4e58e2d016c0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:42.108716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 278/554] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:42.113840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23299d53-4420-4394-a1e1-4e58e2d016c0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:42.136617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef03a06-d06d-4bfd-81b4-24af872b83d7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:42.173821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e5fdf-7134-4335-8f99-5ff878c532fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:42.174848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e5fdf-7134-4335-8f99-5ff878c532fa None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.175402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e5fdf-7134-4335-8f99-5ff878c532fa None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.175681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e5fdf-7134-4335-8f99-5ff878c532fa None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:42.176442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e5fdf-7134-4335-8f99-5ff878c532fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:42.176751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e5fdf-7134-4335-8f99-5ff878c532fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:42.178346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e5fdf-7134-4335-8f99-5ff878c532fa None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:42.187450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 277/555] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:42.188803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ce3ac-5b93-4f3d-a020-e1c1329f90a8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.189091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ce3ac-5b93-4f3d-a020-e1c1329f90a8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.189462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ce3ac-5b93-4f3d-a020-e1c1329f90a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:42.194346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:42.206954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e58ce3ac-5b93-4f3d-a020-e1c1329f90a8 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:42.208719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e58ce3ac-5b93-4f3d-a020-e1c1329f90a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:42.224358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23299d53-4420-4394-a1e1-4e58e2d016c0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:42.261850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71aa76-5b3a-4ab7-88b1-a87453acc8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:42.262651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71aa76-5b3a-4ab7-88b1-a87453acc8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.263118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71aa76-5b3a-4ab7-88b1-a87453acc8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.263398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71aa76-5b3a-4ab7-88b1-a87453acc8f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:42.274767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d71aa76-5b3a-4ab7-88b1-a87453acc8f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:42.276279 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d71aa76-5b3a-4ab7-88b1-a87453acc8f0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:42.282147 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 278/556] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:42.282852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ce3ac-5b93-4f3d-a020-e1c1329f90a8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1d71aa76-5b3a-4ab7-88b1-a87453acc8f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:42.310124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:42.312915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:42.315879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:42.315879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:42.315879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:42.315879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:42.315879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 279/557] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:42.322808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71aa76-5b3a-4ab7-88b1-a87453acc8f0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:42.339851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:42.388541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ce3ac-5b93-4f3d-a020-e1c1329f90a8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:42.420045 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:42.425500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53501965-f105-4f99-96c2-1861d4594dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:42.426349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53501965-f105-4f99-96c2-1861d4594dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.426634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53501965-f105-4f99-96c2-1861d4594dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.426945 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53501965-f105-4f99-96c2-1861d4594dc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:42.439393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53501965-f105-4f99-96c2-1861d4594dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:42.440033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71aa76-5b3a-4ab7-88b1-a87453acc8f0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:42.441217 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53501965-f105-4f99-96c2-1861d4594dc0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-53501965-f105-4f99-96c2-1861d4594dc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:42.460758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:42.463631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:42.465670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:42.465670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:42.465670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:42.465670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:42.465670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 279/558] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:42.465670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53501965-f105-4f99-96c2-1861d4594dc0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:42.480230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18b54a-d0d2-4770-9370-8502c11eae19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:42.480937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18b54a-d0d2-4770-9370-8502c11eae19 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.481286 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18b54a-d0d2-4770-9370-8502c11eae19 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.481570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18b54a-d0d2-4770-9370-8502c11eae19 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:42.482308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18b54a-d0d2-4770-9370-8502c11eae19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:42.482620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18b54a-d0d2-4770-9370-8502c11eae19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:42.484119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18b54a-d0d2-4770-9370-8502c11eae19 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:42.498117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 280/559] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:42.499114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18b54a-d0d2-4770-9370-8502c11eae19 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:42.570565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53501965-f105-4f99-96c2-1861d4594dc0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:42.574507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:42.612478 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18b54a-d0d2-4770-9370-8502c11eae19 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:42.619326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c70c2-a0d8-4274-a6df-34ce47ed649e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:42.619326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c70c2-a0d8-4274-a6df-34ce47ed649e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.619326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c70c2-a0d8-4274-a6df-34ce47ed649e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.619326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c70c2-a0d8-4274-a6df-34ce47ed649e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:42.631754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c70c2-a0d8-4274-a6df-34ce47ed649e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6b7c70c2-a0d8-4274-a6df-34ce47ed649e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:42.642984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:42.644145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:42.645166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:42.646644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372e8c1-61f5-4e5f-896b-11fa639d2c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:42.647436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372e8c1-61f5-4e5f-896b-11fa639d2c1a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.647641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 280/560] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:42.648109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372e8c1-61f5-4e5f-896b-11fa639d2c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.648732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c70c2-a0d8-4274-a6df-34ce47ed649e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:42.649354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372e8c1-61f5-4e5f-896b-11fa639d2c1a None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:42.650190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372e8c1-61f5-4e5f-896b-11fa639d2c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:42.654576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372e8c1-61f5-4e5f-896b-11fa639d2c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:42.656311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372e8c1-61f5-4e5f-896b-11fa639d2c1a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:42.670123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 281/561] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:42.671251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372e8c1-61f5-4e5f-896b-11fa639d2c1a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:42.724627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:42.782709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c70c2-a0d8-4274-a6df-34ce47ed649e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:42.785908 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372e8c1-61f5-4e5f-896b-11fa639d2c1a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:42.816097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:42.822924 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424aae59-f1d9-46c0-9310-e76bd24a7bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:42.823690 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aae59-f1d9-46c0-9310-e76bd24a7bec None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.823971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aae59-f1d9-46c0-9310-e76bd24a7bec None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.824229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aae59-f1d9-46c0-9310-e76bd24a7bec None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:42.825089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424aae59-f1d9-46c0-9310-e76bd24a7bec None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:42.825388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d635055-4e2f-4479-b791-9a3d4fbb2b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:42.825504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424aae59-f1d9-46c0-9310-e76bd24a7bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:42.826194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d635055-4e2f-4479-b791-9a3d4fbb2b5a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.826469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d635055-4e2f-4479-b791-9a3d4fbb2b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.826748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d635055-4e2f-4479-b791-9a3d4fbb2b5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:42.826860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424aae59-f1d9-46c0-9310-e76bd24a7bec None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:42.834412 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d635055-4e2f-4479-b791-9a3d4fbb2b5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:42.840475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 282/562] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:42.840623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d635055-4e2f-4479-b791-9a3d4fbb2b5a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:42.840704 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424aae59-f1d9-46c0-9310-e76bd24a7bec None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-8d635055-4e2f-4479-b791-9a3d4fbb2b5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:42.855875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:42.856775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:42.857659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:42.857659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:42.857659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:42.857659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:42.857659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:42.857659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:42.857659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:42.857659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:42.858072 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 281/563] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:42.859141 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85b0f4-80aa-405d-acb6-b93403f62aef None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.859428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85b0f4-80aa-405d-acb6-b93403f62aef None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.859705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85b0f4-80aa-405d-acb6-b93403f62aef None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:42.861131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 282/564] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:42.861648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85b0f4-80aa-405d-acb6-b93403f62aef None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:42.960641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424aae59-f1d9-46c0-9310-e76bd24a7bec None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:42.983088 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85b0f4-80aa-405d-acb6-b93403f62aef None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:42.986425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:42.997697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4021a-facd-4b03-88b2-1459ec8fcc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:42.998505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4021a-facd-4b03-88b2-1459ec8fcc0f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:42.998814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4021a-facd-4b03-88b2-1459ec8fcc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:42.999087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4021a-facd-4b03-88b2-1459ec8fcc0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.007266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4021a-facd-4b03-88b2-1459ec8fcc0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:43.008715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4021a-facd-4b03-88b2-1459ec8fcc0f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:43.022164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232907a1-c8cc-4a87-bb42-a7af379952e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-38e4021a-facd-4b03-88b2-1459ec8fcc0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:43.022975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:43.023878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:43.024783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:43.024783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:43.024783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:43.024783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:43.024783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:43.024783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:43.024783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:43.024783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:43.024783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:22:43.024783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:43.025942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232907a1-c8cc-4a87-bb42-a7af379952e0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.025942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232907a1-c8cc-4a87-bb42-a7af379952e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.025942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232907a1-c8cc-4a87-bb42-a7af379952e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.026126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 283/565] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jun 29 08:22:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:43.026126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4021a-facd-4b03-88b2-1459ec8fcc0f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:43.038939 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232907a1-c8cc-4a87-bb42-a7af379952e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:43.040401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232907a1-c8cc-4a87-bb42-a7af379952e0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-232907a1-c8cc-4a87-bb42-a7af379952e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:43.059137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:43.060258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:43.069640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:43.069640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:43.069640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:43.069640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:43.069640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 283/566] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:43.069640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538992a0-7bdb-489c-af30-3cf6f5110768 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.069640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538992a0-7bdb-489c-af30-3cf6f5110768 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.069640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538992a0-7bdb-489c-af30-3cf6f5110768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.079823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-538992a0-7bdb-489c-af30-3cf6f5110768 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:43.081108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-538992a0-7bdb-489c-af30-3cf6f5110768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:43.094921 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:43.144121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4021a-facd-4b03-88b2-1459ec8fcc0f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:43.148494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 284/567] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:43.149937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538992a0-7bdb-489c-af30-3cf6f5110768 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:43.187713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a6201-4624-45b3-9571-051f61f728d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:43.188486 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a6201-4624-45b3-9571-051f61f728d0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.188881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a6201-4624-45b3-9571-051f61f728d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.189160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a6201-4624-45b3-9571-051f61f728d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.190018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a6201-4624-45b3-9571-051f61f728d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:43.190330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a6201-4624-45b3-9571-051f61f728d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:43.192083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a6201-4624-45b3-9571-051f61f728d0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:43.209258 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 284/568] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:43.209258 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a6201-4624-45b3-9571-051f61f728d0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:43.247749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:43.265410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538992a0-7bdb-489c-af30-3cf6f5110768 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:43.301004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360743c-480b-4b88-8184-d57d2dc4bc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:43.301687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360743c-480b-4b88-8184-d57d2dc4bc99 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.301972 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360743c-480b-4b88-8184-d57d2dc4bc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.302247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5360743c-480b-4b88-8184-d57d2dc4bc99 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.303186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360743c-480b-4b88-8184-d57d2dc4bc99 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:43.303505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360743c-480b-4b88-8184-d57d2dc4bc99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:43.304960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5360743c-480b-4b88-8184-d57d2dc4bc99 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:43.312157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a6201-4624-45b3-9571-051f61f728d0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:43.322400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 285/569] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jun 29 08:22:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:43.323256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360743c-480b-4b88-8184-d57d2dc4bc99 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:43.332970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:43.355991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3517060-fa2f-42c7-8a60-0d68df05346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:43.356655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3517060-fa2f-42c7-8a60-0d68df05346b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.356930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3517060-fa2f-42c7-8a60-0d68df05346b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.357219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3517060-fa2f-42c7-8a60-0d68df05346b None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.357947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3517060-fa2f-42c7-8a60-0d68df05346b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:43.358273 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3517060-fa2f-42c7-8a60-0d68df05346b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:43.359675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3517060-fa2f-42c7-8a60-0d68df05346b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:43.371171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 285/570] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:43.372229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3517060-fa2f-42c7-8a60-0d68df05346b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:43.436032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5360743c-480b-4b88-8184-d57d2dc4bc99 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:43.474058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae3623-5137-4ae7-af21-fbb5b90d02dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:43.474897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae3623-5137-4ae7-af21-fbb5b90d02dc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.475179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae3623-5137-4ae7-af21-fbb5b90d02dc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.475454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae3623-5137-4ae7-af21-fbb5b90d02dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.475552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3517060-fa2f-42c7-8a60-0d68df05346b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:43.486160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aae3623-5137-4ae7-af21-fbb5b90d02dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0aae3623-5137-4ae7-af21-fbb5b90d02dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:43.494478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:43.495651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:43.496635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 286/571] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:43.497530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae3623-5137-4ae7-af21-fbb5b90d02dc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:43.501400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:43.512207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e863ff-c45b-4e5d-b62d-3855ff1f55df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:43.512950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e863ff-c45b-4e5d-b62d-3855ff1f55df None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.513236 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e863ff-c45b-4e5d-b62d-3855ff1f55df None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.513514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e863ff-c45b-4e5d-b62d-3855ff1f55df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.514362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e863ff-c45b-4e5d-b62d-3855ff1f55df None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=3b38411347754c058eb6640829078e1d) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:43.514675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e863ff-c45b-4e5d-b62d-3855ff1f55df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:43.516153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e863ff-c45b-4e5d-b62d-3855ff1f55df None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:43.521785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 286/572] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:43 2022] GET /identity/v3/role_assignments?user.id=3b38411347754c058eb6640829078e1d&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:43.522551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e863ff-c45b-4e5d-b62d-3855ff1f55df None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:43.565845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:43.616006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae3623-5137-4ae7-af21-fbb5b90d02dc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:43.633904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e863ff-c45b-4e5d-b62d-3855ff1f55df None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:43.653487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86393025-217a-428e-8d67-0598329b6873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:43.654394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86393025-217a-428e-8d67-0598329b6873 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.654771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86393025-217a-428e-8d67-0598329b6873 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.655114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86393025-217a-428e-8d67-0598329b6873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.667754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86393025-217a-428e-8d67-0598329b6873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:43.669354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86393025-217a-428e-8d67-0598329b6873 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:43.673421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e90cc-d8e4-4178-a3c8-04ccfc2470bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:43.674301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e90cc-d8e4-4178-a3c8-04ccfc2470bd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.675306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e90cc-d8e4-4178-a3c8-04ccfc2470bd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.675594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e90cc-d8e4-4178-a3c8-04ccfc2470bd None admin] PATH_INFO: `/v3/users/glance` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.683794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e90cc-d8e4-4178-a3c8-04ccfc2470bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:43.685256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e90cc-d8e4-4178-a3c8-04ccfc2470bd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-86393025-217a-428e-8d67-0598329b6873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:43.689808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:43.690923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:43.692043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:43.692043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:43.692043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:43.692043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:43.697700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 287/573] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:43.697700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d232106-a82c-4443-882c-c8b9635fc72a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.697700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d232106-a82c-4443-882c-c8b9635fc72a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.697700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d232106-a82c-4443-882c-c8b9635fc72a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.697700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 288/574] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:43.697700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d232106-a82c-4443-882c-c8b9635fc72a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ff5e90cc-d8e4-4178-a3c8-04ccfc2470bd None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:43.708887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:43.710033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:43.711306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:43.711306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:43.711306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:43.711306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:43.711306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:43.711306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:43.711306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:43.711306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:43.711306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 08:22:43.711306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:43.716586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 287/575] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:43.716586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e90cc-d8e4-4178-a3c8-04ccfc2470bd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:43.764456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:43.813478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d232106-a82c-4443-882c-c8b9635fc72a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:43.824438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:43.827381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e90cc-d8e4-4178-a3c8-04ccfc2470bd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:43.854818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a1d41-10eb-4b11-a97b-8ed830e63a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:43.855521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a1d41-10eb-4b11-a97b-8ed830e63a66 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.855802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a1d41-10eb-4b11-a97b-8ed830e63a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.856077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a1d41-10eb-4b11-a97b-8ed830e63a66 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.856826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a1d41-10eb-4b11-a97b-8ed830e63a66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:43.857133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a1d41-10eb-4b11-a97b-8ed830e63a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:43.858508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a1d41-10eb-4b11-a97b-8ed830e63a66 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:43.866394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620abea-c96d-42f5-a735-bfafcb6f2bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:43.867108 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620abea-c96d-42f5-a735-bfafcb6f2bff None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.871909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620abea-c96d-42f5-a735-bfafcb6f2bff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.872044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 289/576] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:43.872139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620abea-c96d-42f5-a735-bfafcb6f2bff None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.872139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620abea-c96d-42f5-a735-bfafcb6f2bff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:43.872139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620abea-c96d-42f5-a735-bfafcb6f2bff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:43.872387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c0bad-7904-490f-8bcf-989e159fd0bc None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:43.872387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c0bad-7904-490f-8bcf-989e159fd0bc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:43.872387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c0bad-7904-490f-8bcf-989e159fd0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:43.872601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620abea-c96d-42f5-a735-bfafcb6f2bff None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:43.884320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 288/577] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:43.889411 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9e6c0bad-7904-490f-8bcf-989e159fd0bc None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:43.889548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620abea-c96d-42f5-a735-bfafcb6f2bff None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:43.891662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9e6c0bad-7904-490f-8bcf-989e159fd0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:43.970742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 290/578] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:43.970742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c0bad-7904-490f-8bcf-989e159fd0bc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.012650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620abea-c96d-42f5-a735-bfafcb6f2bff None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.019662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:44.054963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff90bf-f519-4616-8770-a3aba7ef15ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.055642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff90bf-f519-4616-8770-a3aba7ef15ad None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.055928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff90bf-f519-4616-8770-a3aba7ef15ad None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.056202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff90bf-f519-4616-8770-a3aba7ef15ad None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:44.057059 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff90bf-f519-4616-8770-a3aba7ef15ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:44.057352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff90bf-f519-4616-8770-a3aba7ef15ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:44.058863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff90bf-f519-4616-8770-a3aba7ef15ad None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:44.074779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c0bad-7904-490f-8bcf-989e159fd0bc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.076964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 289/579] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jun 29 08:22:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:44.077836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff90bf-f519-4616-8770-a3aba7ef15ad None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.097970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:44.118047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc19fc1-98b1-4212-a8d3-e7d1363813a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.118881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc19fc1-98b1-4212-a8d3-e7d1363813a5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.119167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc19fc1-98b1-4212-a8d3-e7d1363813a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.119443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc19fc1-98b1-4212-a8d3-e7d1363813a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:44.130140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc19fc1-98b1-4212-a8d3-e7d1363813a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:44.131620 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc19fc1-98b1-4212-a8d3-e7d1363813a5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5fc19fc1-98b1-4212-a8d3-e7d1363813a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:44.149961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:44.151519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:44.152408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:44.152408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:44.152408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:44.152408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:44.152408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 291/580] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:44.152408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc19fc1-98b1-4212-a8d3-e7d1363813a5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.189020 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff90bf-f519-4616-8770-a3aba7ef15ad None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.225651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a0a72-6deb-4e81-b998-e6491a7eea3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.226448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a0a72-6deb-4e81-b998-e6491a7eea3c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.226750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a0a72-6deb-4e81-b998-e6491a7eea3c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.227024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a0a72-6deb-4e81-b998-e6491a7eea3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:44.234970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a0a72-6deb-4e81-b998-e6491a7eea3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:44.236418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a0a72-6deb-4e81-b998-e6491a7eea3c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-429a0a72-6deb-4e81-b998-e6491a7eea3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:44.251523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:44.252723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:44.253776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:44.253776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:44.253776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:44.253776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:44.253776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:44.253776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:44.253776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:44.253776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:44.253776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 290/581] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:44.253776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a0a72-6deb-4e81-b998-e6491a7eea3c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.265619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:44.268657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc19fc1-98b1-4212-a8d3-e7d1363813a5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.305400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779aa98-ab58-4cd5-9fe3-356a94667ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.306122 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779aa98-ab58-4cd5-9fe3-356a94667ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.306403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779aa98-ab58-4cd5-9fe3-356a94667ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.306683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779aa98-ab58-4cd5-9fe3-356a94667ec7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:44.317477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779aa98-ab58-4cd5-9fe3-356a94667ec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-9779aa98-ab58-4cd5-9fe3-356a94667ec7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:44.325136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:44.326235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 292/582] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:44.327220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779aa98-ab58-4cd5-9fe3-356a94667ec7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.340975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:44.366518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a0a72-6deb-4e81-b998-e6491a7eea3c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.402727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b925af6-513d-4897-be34-be774e8ac8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.403511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b925af6-513d-4897-be34-be774e8ac8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.403792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b925af6-513d-4897-be34-be774e8ac8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.404065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b925af6-513d-4897-be34-be774e8ac8d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:44.414817 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b925af6-513d-4897-be34-be774e8ac8d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:44.416264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b925af6-513d-4897-be34-be774e8ac8d6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3b925af6-513d-4897-be34-be774e8ac8d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:44.433333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:44.434198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:44.435140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:44.435140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:44.435140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:44.435140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:44.435140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 291/583] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:44.435840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b925af6-513d-4897-be34-be774e8ac8d6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.439642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779aa98-ab58-4cd5-9fe3-356a94667ec7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.480968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c15d-8fa5-406b-a112-456e37176cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.481648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c15d-8fa5-406b-a112-456e37176cff None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.481917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c15d-8fa5-406b-a112-456e37176cff None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.482193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384c15d-8fa5-406b-a112-456e37176cff None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:44.482949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384c15d-8fa5-406b-a112-456e37176cff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:44.483262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384c15d-8fa5-406b-a112-456e37176cff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:44.484665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384c15d-8fa5-406b-a112-456e37176cff None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:44.497477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 293/584] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:44.498183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c15d-8fa5-406b-a112-456e37176cff None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.520534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:44.545923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b925af6-513d-4897-be34-be774e8ac8d6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.573600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:44.582490 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3ea43-8cab-4d99-ad1d-b9df4dd56b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.583185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3ea43-8cab-4d99-ad1d-b9df4dd56b20 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.583457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3ea43-8cab-4d99-ad1d-b9df4dd56b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.583731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3ea43-8cab-4d99-ad1d-b9df4dd56b20 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:44.584527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3ea43-8cab-4d99-ad1d-b9df4dd56b20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:44.584834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3ea43-8cab-4d99-ad1d-b9df4dd56b20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:44.586266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3ea43-8cab-4d99-ad1d-b9df4dd56b20 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:44.599955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 292/585] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:44.600543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3ea43-8cab-4d99-ad1d-b9df4dd56b20 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.610059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384c15d-8fa5-406b-a112-456e37176cff None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.648600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9f279-54d2-4110-99b5-8dffdf1d2b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.649610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9f279-54d2-4110-99b5-8dffdf1d2b3a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.649610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9f279-54d2-4110-99b5-8dffdf1d2b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.649780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9f279-54d2-4110-99b5-8dffdf1d2b3a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:44.650549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9f279-54d2-4110-99b5-8dffdf1d2b3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:44.650898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9f279-54d2-4110-99b5-8dffdf1d2b3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:44.652265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9f279-54d2-4110-99b5-8dffdf1d2b3a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:44.660726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 294/586] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:44.661545 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9f279-54d2-4110-99b5-8dffdf1d2b3a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.717444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3ea43-8cab-4d99-ad1d-b9df4dd56b20 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.747503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:44.753364 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02739b3a-ac79-492e-afdd-0982a858b90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.754038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02739b3a-ac79-492e-afdd-0982a858b90d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.754322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02739b3a-ac79-492e-afdd-0982a858b90d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.755050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02739b3a-ac79-492e-afdd-0982a858b90d None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:44.755814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02739b3a-ac79-492e-afdd-0982a858b90d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:44.756100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02739b3a-ac79-492e-afdd-0982a858b90d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:44.757488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02739b3a-ac79-492e-afdd-0982a858b90d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:44.766452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9f279-54d2-4110-99b5-8dffdf1d2b3a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.769455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 293/587] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:44.770003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02739b3a-ac79-492e-afdd-0982a858b90d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.804699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e0401-a067-40ed-afc5-616a2cef7740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.805503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e0401-a067-40ed-afc5-616a2cef7740 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.805783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e0401-a067-40ed-afc5-616a2cef7740 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.806056 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e0401-a067-40ed-afc5-616a2cef7740 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:44.814046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e0401-a067-40ed-afc5-616a2cef7740 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:44.815562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e0401-a067-40ed-afc5-616a2cef7740 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f08e0401-a067-40ed-afc5-616a2cef7740 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:44.829756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:44.830988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 295/588] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:44.831890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e0401-a067-40ed-afc5-616a2cef7740 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.840296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:44.897269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02739b3a-ac79-492e-afdd-0982a858b90d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.931722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34b07b-8a7c-4883-be9e-f90166de94c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.932848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34b07b-8a7c-4883-be9e-f90166de94c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.933342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34b07b-8a7c-4883-be9e-f90166de94c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.933599 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34b07b-8a7c-4883-be9e-f90166de94c4 None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/7bd500f020f143339c5a4fc9f0728ee4/roles/8f88a54639c64acdbdb152911e5f9eb2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:44.937757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e0401-a067-40ed-afc5-616a2cef7740 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:44.948653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34b07b-8a7c-4883-be9e-f90166de94c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=7bd500f020f143339c5a4fc9f0728ee4, role_id=8f88a54639c64acdbdb152911e5f9eb2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:44.950120 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34b07b-8a7c-4883-be9e-f90166de94c4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:44.964171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 294/589] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:22:44 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/7bd500f020f143339c5a4fc9f0728ee4/roles/8f88a54639c64acdbdb152911e5f9eb2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:44.964994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34b07b-8a7c-4883-be9e-f90166de94c4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:44.988847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d2150-00e5-4e86-884d-2923daf66b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:44.990172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2150-00e5-4e86-884d-2923daf66b49 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:44.990501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2150-00e5-4e86-884d-2923daf66b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:44.990848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2150-00e5-4e86-884d-2923daf66b49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:45.005109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d2150-00e5-4e86-884d-2923daf66b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:45.007257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d2150-00e5-4e86-884d-2923daf66b49 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:45.011637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-fa4d2150-00e5-4e86-884d-2923daf66b49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:45.033348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:45.034467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:45.035415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:45.035415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:45.035415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:45.035415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:45.035415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 296/590] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:45.036395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d2150-00e5-4e86-884d-2923daf66b49 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:45.103220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:45.108578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34b07b-8a7c-4883-be9e-f90166de94c4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:45.182142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d2150-00e5-4e86-884d-2923daf66b49 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:45.187379 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e77bb-f90e-45dc-86eb-df2fda5482bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:45.188481 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e77bb-f90e-45dc-86eb-df2fda5482bf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:45.188971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e77bb-f90e-45dc-86eb-df2fda5482bf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:45.189459 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e77bb-f90e-45dc-86eb-df2fda5482bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:45.205227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e77bb-f90e-45dc-86eb-df2fda5482bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:45.207587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e77bb-f90e-45dc-86eb-df2fda5482bf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:45.224349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44fc2a-4d99-4960-8242-61c9477fa182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:45.225036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44fc2a-4d99-4960-8242-61c9477fa182 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:45.225319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44fc2a-4d99-4960-8242-61c9477fa182 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:45.225593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44fc2a-4d99-4960-8242-61c9477fa182 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:45.226461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44fc2a-4d99-4960-8242-61c9477fa182 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:45.226814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44fc2a-4d99-4960-8242-61c9477fa182 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:45.228323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44fc2a-4d99-4960-8242-61c9477fa182 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-d03e77bb-f90e-45dc-86eb-df2fda5482bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:45.237095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:45.238253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:45.239185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:45.239185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:45.239185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:45.239185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:45.239185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:45.239185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:45.239185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:45.239185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:45.245166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 295/591] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:45.246238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e77bb-f90e-45dc-86eb-df2fda5482bf None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:45.250274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 297/592] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jun 29 08:22:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:45.258653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44fc2a-4d99-4960-8242-61c9477fa182 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:45.342413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:45.439255 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44fc2a-4d99-4960-8242-61c9477fa182 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:45.448025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e77bb-f90e-45dc-86eb-df2fda5482bf None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:45.508626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b7f55-26f3-4231-84b3-1bb2af6d760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:45.516461 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b7f55-26f3-4231-84b3-1bb2af6d760b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:45.519314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b7f55-26f3-4231-84b3-1bb2af6d760b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:45.520521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b7f55-26f3-4231-84b3-1bb2af6d760b None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:45.521109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e83d11-2feb-44cf-b612-344d6ca6cacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:45.521909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e83d11-2feb-44cf-b612-344d6ca6cacf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:45.522191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e83d11-2feb-44cf-b612-344d6ca6cacf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:45.522466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e83d11-2feb-44cf-b612-344d6ca6cacf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:45.523640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b7f55-26f3-4231-84b3-1bb2af6d760b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:45.524247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b7f55-26f3-4231-84b3-1bb2af6d760b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:45.527307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b7f55-26f3-4231-84b3-1bb2af6d760b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:45.536537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e83d11-2feb-44cf-b612-344d6ca6cacf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:45.538592 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e83d11-2feb-44cf-b612-344d6ca6cacf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:45.558444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 296/593] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:45.559357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b7f55-26f3-4231-84b3-1bb2af6d760b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d4e83d11-2feb-44cf-b612-344d6ca6cacf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:45.563307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:45.564235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:45.565499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:45.565499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:45.565499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:45.565499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:45.565499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 298/594] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:45.566224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e44dfe-9c31-409f-8b2f-452cbd6062dc None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:45.566505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e44dfe-9c31-409f-8b2f-452cbd6062dc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:45.571268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e44dfe-9c31-409f-8b2f-452cbd6062dc None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:45.576665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 299/595] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:45.576665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e44dfe-9c31-409f-8b2f-452cbd6062dc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:45.586862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:45.678952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b7f55-26f3-4231-84b3-1bb2af6d760b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:45.694482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e44dfe-9c31-409f-8b2f-452cbd6062dc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:45.697774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:45.714639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d0aad-5d1f-4d98-a7e7-c9a62270f41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:45.715444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d0aad-5d1f-4d98-a7e7-c9a62270f41f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:45.715825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d0aad-5d1f-4d98-a7e7-c9a62270f41f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:45.716201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d0aad-5d1f-4d98-a7e7-c9a62270f41f None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:45.717092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d0aad-5d1f-4d98-a7e7-c9a62270f41f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:45.717519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d0aad-5d1f-4d98-a7e7-c9a62270f41f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:45.719082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d0aad-5d1f-4d98-a7e7-c9a62270f41f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:45.735765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5db20-dfc1-4d83-90ea-f46ac294581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:45.735945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 297/596] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:45.736040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5db20-dfc1-4d83-90ea-f46ac294581b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:45.736124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d0aad-5d1f-4d98-a7e7-c9a62270f41f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:45.736230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5db20-dfc1-4d83-90ea-f46ac294581b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:45.736230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5db20-dfc1-4d83-90ea-f46ac294581b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:45.743971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5db20-dfc1-4d83-90ea-f46ac294581b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:45.745660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5db20-dfc1-4d83-90ea-f46ac294581b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f1e5db20-dfc1-4d83-90ea-f46ac294581b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:45.760322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:45.761509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:45.762631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:45.762631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:45.762631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:45.762631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:45.762631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:45.762631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:45.762631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:45.762631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:45.762631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 08:22:45.762631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:45.763273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 300/597] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:45.768962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dd0cd-9d9d-452a-a33a-0c7e1923e794 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:45.769247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dd0cd-9d9d-452a-a33a-0c7e1923e794 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:45.769521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dd0cd-9d9d-452a-a33a-0c7e1923e794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:45.787000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-944dd0cd-9d9d-452a-a33a-0c7e1923e794 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:45.788220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-944dd0cd-9d9d-452a-a33a-0c7e1923e794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:45.822584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:45.847055 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d0aad-5d1f-4d98-a7e7-c9a62270f41f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:45.865586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 301/598] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:45.866203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dd0cd-9d9d-452a-a33a-0c7e1923e794 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:45.890247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb9c57-3b87-4ca5-a98a-f82886e5e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:45.891222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb9c57-3b87-4ca5-a98a-f82886e5e876 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:45.891609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb9c57-3b87-4ca5-a98a-f82886e5e876 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:45.891951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb9c57-3b87-4ca5-a98a-f82886e5e876 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:45.892853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb9c57-3b87-4ca5-a98a-f82886e5e876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:45.893351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb9c57-3b87-4ca5-a98a-f82886e5e876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:45.894895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb9c57-3b87-4ca5-a98a-f82886e5e876 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:45.907799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 298/599] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:45.912516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb9c57-3b87-4ca5-a98a-f82886e5e876 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:45.967192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:45.974629 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dd0cd-9d9d-452a-a33a-0c7e1923e794 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:46.010129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5cf18-07ef-4530-8104-76c2479a41d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:46.011062 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5cf18-07ef-4530-8104-76c2479a41d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:46.011339 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5cf18-07ef-4530-8104-76c2479a41d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:46.011609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5cf18-07ef-4530-8104-76c2479a41d4 None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/fd09453b5db645d7b03522f91effb8d4/roles/8f88a54639c64acdbdb152911e5f9eb2` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:46.020739 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb9c57-3b87-4ca5-a98a-f82886e5e876 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:46.026351 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5cf18-07ef-4530-8104-76c2479a41d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=fd09453b5db645d7b03522f91effb8d4, role_id=8f88a54639c64acdbdb152911e5f9eb2)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:46.027863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5cf18-07ef-4530-8104-76c2479a41d4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:46.050726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 302/600] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:22:45 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/fd09453b5db645d7b03522f91effb8d4/roles/8f88a54639c64acdbdb152911e5f9eb2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:46.050726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5cf18-07ef-4530-8104-76c2479a41d4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:46.069358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201b9c7-5c40-43b0-9b23-2dab6ef5a5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:46.070104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201b9c7-5c40-43b0-9b23-2dab6ef5a5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:46.070383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201b9c7-5c40-43b0-9b23-2dab6ef5a5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:46.070800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201b9c7-5c40-43b0-9b23-2dab6ef5a5c1 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:46.071770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201b9c7-5c40-43b0-9b23-2dab6ef5a5c1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:46.072088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201b9c7-5c40-43b0-9b23-2dab6ef5a5c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:46.073810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201b9c7-5c40-43b0-9b23-2dab6ef5a5c1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:46.086042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:46.107196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 299/601] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Jun 29 08:22:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:46.107196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201b9c7-5c40-43b0-9b23-2dab6ef5a5c1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:46.182740 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5cf18-07ef-4530-8104-76c2479a41d4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:46.224615 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceffb3f-0699-4760-92a9-52c7f0dfc611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:46.225710 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceffb3f-0699-4760-92a9-52c7f0dfc611 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:46.225710 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceffb3f-0699-4760-92a9-52c7f0dfc611 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:46.225984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceffb3f-0699-4760-92a9-52c7f0dfc611 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:46.239519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceffb3f-0699-4760-92a9-52c7f0dfc611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0ceffb3f-0699-4760-92a9-52c7f0dfc611 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:46.247623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:46.248625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 303/602] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:46.250110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceffb3f-0699-4760-92a9-52c7f0dfc611 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:46.255743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:46.268004 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201b9c7-5c40-43b0-9b23-2dab6ef5a5c1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:46.311598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f8668-6d31-421f-86c5-36d3df82d41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:46.312919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8668-6d31-421f-86c5-36d3df82d41c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:46.313351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8668-6d31-421f-86c5-36d3df82d41c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:46.313885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f8668-6d31-421f-86c5-36d3df82d41c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:46.323542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f8668-6d31-421f-86c5-36d3df82d41c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:46.325363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f8668-6d31-421f-86c5-36d3df82d41c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c80f8668-6d31-421f-86c5-36d3df82d41c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:46.343994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:46.345101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:46.346209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:46.346209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:46.346209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:46.346209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:46.346209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:46.346209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:46.346209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:46.346209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:46.348952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 300/603] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:46.349991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f8668-6d31-421f-86c5-36d3df82d41c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:46.364596 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceffb3f-0699-4760-92a9-52c7f0dfc611 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:46.400952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc6f54-8a0c-4d01-a362-d5a794cd5cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:46.401910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc6f54-8a0c-4d01-a362-d5a794cd5cbd None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:46.402536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc6f54-8a0c-4d01-a362-d5a794cd5cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:46.402536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc6f54-8a0c-4d01-a362-d5a794cd5cbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:46.414016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc6f54-8a0c-4d01-a362-d5a794cd5cbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:46.415734 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc6f54-8a0c-4d01-a362-d5a794cd5cbd None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:46.423025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-83bc6f54-8a0c-4d01-a362-d5a794cd5cbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:46.433782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:46.435377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:46.436276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:46.436276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:46.436276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:46.436276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:46.436690 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 304/604] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:46.438193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc6f54-8a0c-4d01-a362-d5a794cd5cbd None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:46.476771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f8668-6d31-421f-86c5-36d3df82d41c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:46.516052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1953f95-7ee4-4b71-ae53-7e15ea79b559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:46.516868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1953f95-7ee4-4b71-ae53-7e15ea79b559 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:46.517644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1953f95-7ee4-4b71-ae53-7e15ea79b559 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:46.517644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1953f95-7ee4-4b71-ae53-7e15ea79b559 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:46.518820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1953f95-7ee4-4b71-ae53-7e15ea79b559 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:46.519222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1953f95-7ee4-4b71-ae53-7e15ea79b559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:46.520782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1953f95-7ee4-4b71-ae53-7e15ea79b559 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:46.522658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:46.532828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 301/605] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:46.534301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1953f95-7ee4-4b71-ae53-7e15ea79b559 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:46.585683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc6f54-8a0c-4d01-a362-d5a794cd5cbd None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:46.636464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91d2ec-ce50-4d9f-a6d7-ddb42a1d8da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:46.637147 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91d2ec-ce50-4d9f-a6d7-ddb42a1d8da3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:46.637433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91d2ec-ce50-4d9f-a6d7-ddb42a1d8da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:46.637708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91d2ec-ce50-4d9f-a6d7-ddb42a1d8da3 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:46.638789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91d2ec-ce50-4d9f-a6d7-ddb42a1d8da3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:46.639113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91d2ec-ce50-4d9f-a6d7-ddb42a1d8da3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:46.641257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91d2ec-ce50-4d9f-a6d7-ddb42a1d8da3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:46.649978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1953f95-7ee4-4b71-ae53-7e15ea79b559 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:46.658663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 305/606] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:46.660190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91d2ec-ce50-4d9f-a6d7-ddb42a1d8da3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:46.670057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:46.690199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d980f0-5d2a-423a-b4d8-9bfd256ff629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:46.691262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d980f0-5d2a-423a-b4d8-9bfd256ff629 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:46.691792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d980f0-5d2a-423a-b4d8-9bfd256ff629 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:46.692350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d980f0-5d2a-423a-b4d8-9bfd256ff629 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:46.693418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d980f0-5d2a-423a-b4d8-9bfd256ff629 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:46.693954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d980f0-5d2a-423a-b4d8-9bfd256ff629 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:46.695673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d980f0-5d2a-423a-b4d8-9bfd256ff629 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:46.709494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 302/607] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:46.710735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d980f0-5d2a-423a-b4d8-9bfd256ff629 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:46.769662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91d2ec-ce50-4d9f-a6d7-ddb42a1d8da3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:46.787697 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:46.806958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd28b6-8a6a-40a1-b4b3-4ec40cf92897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:46.807769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd28b6-8a6a-40a1-b4b3-4ec40cf92897 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:46.808048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd28b6-8a6a-40a1-b4b3-4ec40cf92897 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:46.808318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd28b6-8a6a-40a1-b4b3-4ec40cf92897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:46.819844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd28b6-8a6a-40a1-b4b3-4ec40cf92897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:46.821375 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd28b6-8a6a-40a1-b4b3-4ec40cf92897 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:46.822721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d980f0-5d2a-423a-b4d8-9bfd256ff629 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-effd28b6-8a6a-40a1-b4b3-4ec40cf92897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:46.840819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:46.841772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:46.842750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:46.842750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:46.842750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:46.842750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:46.846857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 306/608] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:46.847840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd28b6-8a6a-40a1-b4b3-4ec40cf92897 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:46.864314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47bf6c-662b-45be-87fe-ddee51edcbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:46.865618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47bf6c-662b-45be-87fe-ddee51edcbbb None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:46.866155 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47bf6c-662b-45be-87fe-ddee51edcbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:46.866626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47bf6c-662b-45be-87fe-ddee51edcbbb None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/b60ca33edf5a47fdab6e39cc98638c6d/roles/8f88a54639c64acdbdb152911e5f9eb2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:46.883413 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47bf6c-662b-45be-87fe-ddee51edcbbb None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=b60ca33edf5a47fdab6e39cc98638c6d, role_id=8f88a54639c64acdbdb152911e5f9eb2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:46.885360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47bf6c-662b-45be-87fe-ddee51edcbbb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:46.905372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 303/609] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:22:46 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/b60ca33edf5a47fdab6e39cc98638c6d/roles/8f88a54639c64acdbdb152911e5f9eb2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:46.906617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47bf6c-662b-45be-87fe-ddee51edcbbb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:46.943381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:47.005555 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd28b6-8a6a-40a1-b4b3-4ec40cf92897 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:47.036289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47bf6c-662b-45be-87fe-ddee51edcbbb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:47.088701 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c84a28-6235-4681-99d2-8758204c9ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:47.091551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c84a28-6235-4681-99d2-8758204c9ded None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:47.092961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c84a28-6235-4681-99d2-8758204c9ded None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:47.093862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c84a28-6235-4681-99d2-8758204c9ded None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:47.095424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c84a28-6235-4681-99d2-8758204c9ded None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:47.096641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c84a28-6235-4681-99d2-8758204c9ded None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:47.099499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c84a28-6235-4681-99d2-8758204c9ded None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:47.102803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006db08c-db45-437f-b5fb-3f2b10b38a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:47.103930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006db08c-db45-437f-b5fb-3f2b10b38a56 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:47.104413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006db08c-db45-437f-b5fb-3f2b10b38a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:47.104888 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006db08c-db45-437f-b5fb-3f2b10b38a56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:47.122594 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006db08c-db45-437f-b5fb-3f2b10b38a56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:47.125510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006db08c-db45-437f-b5fb-3f2b10b38a56 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:47.129316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 304/610] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:47.134057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c84a28-6235-4681-99d2-8758204c9ded None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-006db08c-db45-437f-b5fb-3f2b10b38a56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:47.155140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:47.156081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:47.156949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:47.156949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:47.156949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:47.156949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:47.156949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:47.156949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:47.156949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:47.156949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:47.160293 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 307/611] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:47.165789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006db08c-db45-437f-b5fb-3f2b10b38a56 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:47.201893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:47.271607 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:47.285333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c84a28-6235-4681-99d2-8758204c9ded None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:47.292238 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006db08c-db45-437f-b5fb-3f2b10b38a56 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:47.326589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d39e7-7141-43f2-99a2-559d55e4af00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:47.327689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d39e7-7141-43f2-99a2-559d55e4af00 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:47.328353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d39e7-7141-43f2-99a2-559d55e4af00 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:47.328812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d39e7-7141-43f2-99a2-559d55e4af00 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:47.333612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f0e05-e3f2-44fb-aa6a-0476672aed0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:47.334346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f0e05-e3f2-44fb-aa6a-0476672aed0a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:47.334709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f0e05-e3f2-44fb-aa6a-0476672aed0a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:47.335065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f0e05-e3f2-44fb-aa6a-0476672aed0a None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:47.335931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f0e05-e3f2-44fb-aa6a-0476672aed0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:47.336313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f0e05-e3f2-44fb-aa6a-0476672aed0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:47.337430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d39e7-7141-43f2-99a2-559d55e4af00 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:47.338047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f0e05-e3f2-44fb-aa6a-0476672aed0a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:47.339285 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d39e7-7141-43f2-99a2-559d55e4af00 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:47.352206 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 308/612] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:47.353534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e2734-b3ab-4e13-b306-e35a0c255a0a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:47.353824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e2734-b3ab-4e13-b306-e35a0c255a0a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:47.354098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e2734-b3ab-4e13-b306-e35a0c255a0a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:47.355633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 309/613] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:47.360250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e2734-b3ab-4e13-b306-e35a0c255a0a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-018d39e7-7141-43f2-99a2-559d55e4af00 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:47.361370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:47.362467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:47.363432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:47.363432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:47.363432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:47.363432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:47.363432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:47.363432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:47.363432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:47.363432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:47.363432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:22:47.363432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:47.363432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 305/614] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:22:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:47.370724 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb6ed2-1116-4fe8-8b60-c4559b53fd26 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:47.370724 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb6ed2-1116-4fe8-8b60-c4559b53fd26 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:47.370724 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb6ed2-1116-4fe8-8b60-c4559b53fd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:47.387337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4efb6ed2-1116-4fe8-8b60-c4559b53fd26 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:47.388802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4efb6ed2-1116-4fe8-8b60-c4559b53fd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:47.477623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:47.481896 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 306/615] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:47.483138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb6ed2-1116-4fe8-8b60-c4559b53fd26 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:47.507133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e2734-b3ab-4e13-b306-e35a0c255a0a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:47.554415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2809b1b-5ec9-427f-92d9-3917220b6e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:47.555268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2809b1b-5ec9-427f-92d9-3917220b6e4b None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:47.555550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2809b1b-5ec9-427f-92d9-3917220b6e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:47.555826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2809b1b-5ec9-427f-92d9-3917220b6e4b None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/49a1439029fa4b719bc114c3f59e0050/roles/8f88a54639c64acdbdb152911e5f9eb2` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:47.575167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2809b1b-5ec9-427f-92d9-3917220b6e4b None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=49a1439029fa4b719bc114c3f59e0050, role_id=8f88a54639c64acdbdb152911e5f9eb2)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:47.577490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2809b1b-5ec9-427f-92d9-3917220b6e4b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:47.591783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:47.606725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 310/616] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:22:47 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/49a1439029fa4b719bc114c3f59e0050/roles/8f88a54639c64acdbdb152911e5f9eb2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:47.606725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2809b1b-5ec9-427f-92d9-3917220b6e4b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:47.635099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb6ed2-1116-4fe8-8b60-c4559b53fd26 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:47.712620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8020a-a0d6-41c2-8b26-41f504235bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:47.713695 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8020a-a0d6-41c2-8b26-41f504235bce None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:47.714015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8020a-a0d6-41c2-8b26-41f504235bce None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:47.714324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8020a-a0d6-41c2-8b26-41f504235bce None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:47.715643 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8020a-a0d6-41c2-8b26-41f504235bce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:47.716225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8020a-a0d6-41c2-8b26-41f504235bce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:47.718719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8020a-a0d6-41c2-8b26-41f504235bce None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:47.720771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2809b1b-5ec9-427f-92d9-3917220b6e4b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:47.756275 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 307/617] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jun 29 08:22:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:47.760446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0c762-6132-46f2-80f1-af77782377b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:47.761281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0c762-6132-46f2-80f1-af77782377b0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:47.761568 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0c762-6132-46f2-80f1-af77782377b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:47.761846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0c762-6132-46f2-80f1-af77782377b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:47.762440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8020a-a0d6-41c2-8b26-41f504235bce None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:47.770613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:47.775533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0c762-6132-46f2-80f1-af77782377b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-57d0c762-6132-46f2-80f1-af77782377b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:47.784870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:47.785841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:47.786708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:47.789870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 311/618] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:47.796494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0c762-6132-46f2-80f1-af77782377b0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:47.910440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8020a-a0d6-41c2-8b26-41f504235bce None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:47.935924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0c762-6132-46f2-80f1-af77782377b0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:47.953563 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cb1c0-0862-400f-922c-522a7864870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:47.954802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cb1c0-0862-400f-922c-522a7864870f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:47.955153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cb1c0-0862-400f-922c-522a7864870f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:47.955521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cb1c0-0862-400f-922c-522a7864870f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:47.959139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:47.969962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cb1c0-0862-400f-922c-522a7864870f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:47.972312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4cb1c0-0862-400f-922c-522a7864870f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:47.982400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1408f-b073-4b59-85b1-488305856d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:47.983928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1408f-b073-4b59-85b1-488305856d0b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:47.984495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1408f-b073-4b59-85b1-488305856d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:47.984951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1408f-b073-4b59-85b1-488305856d0b None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:47.986017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1408f-b073-4b59-85b1-488305856d0b None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:47.986507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1408f-b073-4b59-85b1-488305856d0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:47.988320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1408f-b073-4b59-85b1-488305856d0b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1d4cb1c0-0862-400f-922c-522a7864870f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:47.993410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:47.994455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:47.995367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:47.995367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:47.995367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:47.995367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:47.995367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 308/619] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:48.000394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 312/620] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:48.003668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cb1c0-0862-400f-922c-522a7864870f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:48.005303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1408f-b073-4b59-85b1-488305856d0b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:48.084748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:48.148124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1408f-b073-4b59-85b1-488305856d0b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:48.180278 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4cb1c0-0862-400f-922c-522a7864870f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:48.217799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f24cb-4e9a-4ca1-bb13-f326946c47a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:48.218885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f24cb-4e9a-4ca1-bb13-f326946c47a8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:48.219396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f24cb-4e9a-4ca1-bb13-f326946c47a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:48.219868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f24cb-4e9a-4ca1-bb13-f326946c47a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:48.220799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f24cb-4e9a-4ca1-bb13-f326946c47a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:48.221375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f24cb-4e9a-4ca1-bb13-f326946c47a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:48.223158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f24cb-4e9a-4ca1-bb13-f326946c47a8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:48.236038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 309/621] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:48.243215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e0446-a8c1-4827-8097-77733ae48aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:48.243215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e0446-a8c1-4827-8097-77733ae48aef None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:48.243215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e0446-a8c1-4827-8097-77733ae48aef None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:48.243215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e0446-a8c1-4827-8097-77733ae48aef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:48.245131 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f24cb-4e9a-4ca1-bb13-f326946c47a8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:48.256765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e0446-a8c1-4827-8097-77733ae48aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:48.258367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e0446-a8c1-4827-8097-77733ae48aef None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-c58e0446-a8c1-4827-8097-77733ae48aef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:48.276668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:48.277590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:48.278484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:48.278484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:48.278484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:48.278484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:48.278484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 313/622] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:48.279359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4d855-a61d-4479-b61b-d0e885034f2e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:48.279542 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4d855-a61d-4479-b61b-d0e885034f2e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:48.282712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4d855-a61d-4479-b61b-d0e885034f2e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:48.282712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 314/623] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:48.284531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4d855-a61d-4479-b61b-d0e885034f2e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:48.308111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:48.370828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f24cb-4e9a-4ca1-bb13-f326946c47a8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:48.395253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4d855-a61d-4479-b61b-d0e885034f2e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:48.407301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c56161-b579-43b5-8892-574e25e9f5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:48.408388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c56161-b579-43b5-8892-574e25e9f5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:48.408922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c56161-b579-43b5-8892-574e25e9f5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:48.409518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c56161-b579-43b5-8892-574e25e9f5b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:48.417168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c56161-b579-43b5-8892-574e25e9f5b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:48.418893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c56161-b579-43b5-8892-574e25e9f5b1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:48.427089 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:48.431518 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e65d4-056b-4ca8-8bbe-1990f6295c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:48.432291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e65d4-056b-4ca8-8bbe-1990f6295c47 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:48.432666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e65d4-056b-4ca8-8bbe-1990f6295c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:48.433040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e65d4-056b-4ca8-8bbe-1990f6295c47 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:48.433868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e65d4-056b-4ca8-8bbe-1990f6295c47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:48.434281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e65d4-056b-4ca8-8bbe-1990f6295c47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-37c56161-b579-43b5-8892-574e25e9f5b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:48.434952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:48.436033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:48.437127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:48.437127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:48.437127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:48.437127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:48.437127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:48.437127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:48.437127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:48.437127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:48.438116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e65d4-056b-4ca8-8bbe-1990f6295c47 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:48.443933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 310/624] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:48.445408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c0a2-5d15-4a92-a41a-e170a7694ede None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:48.445862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c0a2-5d15-4a92-a41a-e170a7694ede None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:48.446327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c0a2-5d15-4a92-a41a-e170a7694ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:48.455926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 315/625] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:48.456966 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e65d4-056b-4ca8-8bbe-1990f6295c47 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:48.465370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c087c0a2-5d15-4a92-a41a-e170a7694ede None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:48.466769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c087c0a2-5d15-4a92-a41a-e170a7694ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:48.535794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 311/626] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:48.537015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c0a2-5d15-4a92-a41a-e170a7694ede None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:48.579104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:48.588055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e65d4-056b-4ca8-8bbe-1990f6295c47 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:48.628299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcf021-bee6-4a31-bd7c-b39237c39515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:48.628999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcf021-bee6-4a31-bd7c-b39237c39515 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:48.629345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcf021-bee6-4a31-bd7c-b39237c39515 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:48.629692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcf021-bee6-4a31-bd7c-b39237c39515 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:48.630561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcf021-bee6-4a31-bd7c-b39237c39515 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:48.630974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcf021-bee6-4a31-bd7c-b39237c39515 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:48.632590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcf021-bee6-4a31-bd7c-b39237c39515 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:48.641680 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c0a2-5d15-4a92-a41a-e170a7694ede None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:48.646666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 316/627] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:48.652198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcf021-bee6-4a31-bd7c-b39237c39515 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:48.682679 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8322-867d-4c7d-b5b2-35d228274ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:48.683769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8322-867d-4c7d-b5b2-35d228274ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:48.684285 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8322-867d-4c7d-b5b2-35d228274ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:48.684729 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8322-867d-4c7d-b5b2-35d228274ca5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:48.692804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b8322-867d-4c7d-b5b2-35d228274ca5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:48.694513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b8322-867d-4c7d-b5b2-35d228274ca5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8f5b8322-867d-4c7d-b5b2-35d228274ca5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:48.709597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:48.710548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:48.711603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:48.711603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:48.711603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:48.711603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:48.711603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:48.711603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:48.711603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:48.711603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:48.711603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:22:48.711603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:48.714661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 312/628] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jun 29 08:22:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:48.715702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8322-867d-4c7d-b5b2-35d228274ca5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:48.720157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:48.762268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcf021-bee6-4a31-bd7c-b39237c39515 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:48.799797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e38c4-bdfa-4a6f-9f4f-90eae433bfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:48.801102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e38c4-bdfa-4a6f-9f4f-90eae433bfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:48.801525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e38c4-bdfa-4a6f-9f4f-90eae433bfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:48.801940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e38c4-bdfa-4a6f-9f4f-90eae433bfd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:48.813488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e38c4-bdfa-4a6f-9f4f-90eae433bfd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:48.818900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:48.822846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8322-867d-4c7d-b5b2-35d228274ca5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-639e38c4-bdfa-4a6f-9f4f-90eae433bfd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:48.823599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:48.824692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:48.825892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:48.833893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 317/629] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:48.835034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e38c4-bdfa-4a6f-9f4f-90eae433bfd6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:48.862342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16247912-49ff-4a54-a96f-a7bc9f0b4ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:48.863491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16247912-49ff-4a54-a96f-a7bc9f0b4ea2 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:48.863867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16247912-49ff-4a54-a96f-a7bc9f0b4ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:48.864266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16247912-49ff-4a54-a96f-a7bc9f0b4ea2 None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/3b38411347754c058eb6640829078e1d/roles/8f88a54639c64acdbdb152911e5f9eb2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:48.880459 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16247912-49ff-4a54-a96f-a7bc9f0b4ea2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=3b38411347754c058eb6640829078e1d, role_id=8f88a54639c64acdbdb152911e5f9eb2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:48.882045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16247912-49ff-4a54-a96f-a7bc9f0b4ea2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:48.899728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 313/630] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:22:48 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/3b38411347754c058eb6640829078e1d/roles/8f88a54639c64acdbdb152911e5f9eb2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:22:48.903731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16247912-49ff-4a54-a96f-a7bc9f0b4ea2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:48.980657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:49.000586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e38c4-bdfa-4a6f-9f4f-90eae433bfd6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:49.073718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603b936-6bbf-4422-8dfb-7a760d9b89d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:49.074457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603b936-6bbf-4422-8dfb-7a760d9b89d8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:49.076703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603b936-6bbf-4422-8dfb-7a760d9b89d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:49.077025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603b936-6bbf-4422-8dfb-7a760d9b89d8 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:49.077911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603b936-6bbf-4422-8dfb-7a760d9b89d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:49.078228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603b936-6bbf-4422-8dfb-7a760d9b89d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:49.079732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603b936-6bbf-4422-8dfb-7a760d9b89d8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:49.113895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16247912-49ff-4a54-a96f-a7bc9f0b4ea2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:49.122817 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 318/631] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jun 29 08:22:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:49.125009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603b936-6bbf-4422-8dfb-7a760d9b89d8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:49.157222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3bfa3-25f6-4afc-80df-361ee7199ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:49.157903 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3bfa3-25f6-4afc-80df-361ee7199ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:49.158115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3bfa3-25f6-4afc-80df-361ee7199ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:49.159479 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3bfa3-25f6-4afc-80df-361ee7199ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:49.160261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3bfa3-25f6-4afc-80df-361ee7199ac8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:49.160573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3bfa3-25f6-4afc-80df-361ee7199ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:49.161960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3bfa3-25f6-4afc-80df-361ee7199ac8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:49.175198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 314/632] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:49.180129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3bfa3-25f6-4afc-80df-361ee7199ac8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:49.196765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:49.255464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603b936-6bbf-4422-8dfb-7a760d9b89d8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:49.297969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77488f-84ec-46a0-9573-1d0667c3684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:49.299102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77488f-84ec-46a0-9573-1d0667c3684b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:49.299561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77488f-84ec-46a0-9573-1d0667c3684b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:49.300106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77488f-84ec-46a0-9573-1d0667c3684b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:49.326317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:49.359756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3bfa3-25f6-4afc-80df-361ee7199ac8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:49.389287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77488f-84ec-46a0-9573-1d0667c3684b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:49.391103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77488f-84ec-46a0-9573-1d0667c3684b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:49.417534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a9322-36b8-49f0-b67c-30918f6c6fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:49.419104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a9322-36b8-49f0-b67c-30918f6c6fad None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:49.419727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a9322-36b8-49f0-b67c-30918f6c6fad None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:49.420097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a9322-36b8-49f0-b67c-30918f6c6fad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:49.437072 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a9322-36b8-49f0-b67c-30918f6c6fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:49.440443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a9322-36b8-49f0-b67c-30918f6c6fad None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-db9a9322-36b8-49f0-b67c-30918f6c6fad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:49.467942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:49.468982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:49.470431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:49.470431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:49.470431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:49.470431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:49.470431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 315/633] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:49.470431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c291d-258e-47c5-88b2-b3379b69b4d0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:49.470431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c291d-258e-47c5-88b2-b3379b69b4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:49.471095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c291d-258e-47c5-88b2-b3379b69b4d0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:49.473990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 316/634] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:49.478339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eef2c7-a5ea-4f86-b5dd-769dbd519d36 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:49.478897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eef2c7-a5ea-4f86-b5dd-769dbd519d36 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:49.479389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eef2c7-a5ea-4f86-b5dd-769dbd519d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:49.500149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-54eef2c7-a5ea-4f86-b5dd-769dbd519d36 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:49.502327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-54eef2c7-a5ea-4f86-b5dd-769dbd519d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:49.531903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 319/635] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 08:22:49.532873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77488f-84ec-46a0-9573-1d0667c3684b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:49.571546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:49.590805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 317/636] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:49.592009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eef2c7-a5ea-4f86-b5dd-769dbd519d36 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:49.599476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:49.663160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77488f-84ec-46a0-9573-1d0667c3684b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:49.701058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77dc7c-4f68-4049-bf1d-2af703b1dc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:49.702073 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77dc7c-4f68-4049-bf1d-2af703b1dc31 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:49.702493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77dc7c-4f68-4049-bf1d-2af703b1dc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:49.702609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77dc7c-4f68-4049-bf1d-2af703b1dc31 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:49.703996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77dc7c-4f68-4049-bf1d-2af703b1dc31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:49.704417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77dc7c-4f68-4049-bf1d-2af703b1dc31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:49.706423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77dc7c-4f68-4049-bf1d-2af703b1dc31 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:49.719898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 320/637] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:49.720517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77dc7c-4f68-4049-bf1d-2af703b1dc31 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:49.720970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eef2c7-a5ea-4f86-b5dd-769dbd519d36 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:49.758397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daea97-4947-4a58-8ee1-9f3fcc9084bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:49.759531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daea97-4947-4a58-8ee1-9f3fcc9084bb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:49.759957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daea97-4947-4a58-8ee1-9f3fcc9084bb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:49.760371 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daea97-4947-4a58-8ee1-9f3fcc9084bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:49.773329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09daea97-4947-4a58-8ee1-9f3fcc9084bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:49.775198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09daea97-4947-4a58-8ee1-9f3fcc9084bb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-09daea97-4947-4a58-8ee1-9f3fcc9084bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:49.795586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:49.796631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:49.797598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:49.797598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:49.797598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:49.797598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:49.799940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 318/638] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:49.801047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daea97-4947-4a58-8ee1-9f3fcc9084bb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:49.827844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77dc7c-4f68-4049-bf1d-2af703b1dc31 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:49.839201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:49.856654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:49.865650 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f979f3-bbbb-43ce-aa3a-1025df23cf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:49.866683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f979f3-bbbb-43ce-aa3a-1025df23cf71 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:49.867423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f979f3-bbbb-43ce-aa3a-1025df23cf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:49.867423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f979f3-bbbb-43ce-aa3a-1025df23cf71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:49.881128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f979f3-bbbb-43ce-aa3a-1025df23cf71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-70f979f3-bbbb-43ce-aa3a-1025df23cf71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:49.889123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:49.890108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:49.890978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 321/639] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:49.891653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f979f3-bbbb-43ce-aa3a-1025df23cf71 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:49.927989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daea97-4947-4a58-8ee1-9f3fcc9084bb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:49.972682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62feaf01-ff92-431b-8319-ae8e24d12296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:49.973898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62feaf01-ff92-431b-8319-ae8e24d12296 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:49.974389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62feaf01-ff92-431b-8319-ae8e24d12296 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:49.974893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62feaf01-ff92-431b-8319-ae8e24d12296 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:49.983404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62feaf01-ff92-431b-8319-ae8e24d12296 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:49.985243 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62feaf01-ff92-431b-8319-ae8e24d12296 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-62feaf01-ff92-431b-8319-ae8e24d12296 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:50.002629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:50.003722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:50.004642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:50.004642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:50.004642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:50.004642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:50.004642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:50.004642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:50.004642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:50.004642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:50.004642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 08:22:50.004642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:50.007471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 319/640] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:50.008566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62feaf01-ff92-431b-8319-ae8e24d12296 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:50.013467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f979f3-bbbb-43ce-aa3a-1025df23cf71 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:50.049356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e26d35-431b-4352-a37c-8eb00b31b7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:50.049940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e26d35-431b-4352-a37c-8eb00b31b7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.050243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e26d35-431b-4352-a37c-8eb00b31b7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.050563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e26d35-431b-4352-a37c-8eb00b31b7a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.051326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e26d35-431b-4352-a37c-8eb00b31b7a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:50.051635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e26d35-431b-4352-a37c-8eb00b31b7a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:50.053089 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e26d35-431b-4352-a37c-8eb00b31b7a5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:50.071170 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 322/641] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:50.071170 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e26d35-431b-4352-a37c-8eb00b31b7a5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:50.105882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:50.124130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62feaf01-ff92-431b-8319-ae8e24d12296 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:50.131976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:50.167535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe678d80-29c7-4510-acff-f5d807ce9a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:50.168329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe678d80-29c7-4510-acff-f5d807ce9a38 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.168611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe678d80-29c7-4510-acff-f5d807ce9a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.168885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe678d80-29c7-4510-acff-f5d807ce9a38 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.169635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe678d80-29c7-4510-acff-f5d807ce9a38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:50.169941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe678d80-29c7-4510-acff-f5d807ce9a38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:50.171590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe678d80-29c7-4510-acff-f5d807ce9a38 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:50.181199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 320/642] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:50.182537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6986d1a-e725-470c-9cf4-23ebd859099d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.182858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6986d1a-e725-470c-9cf4-23ebd859099d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.183153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6986d1a-e725-470c-9cf4-23ebd859099d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.189410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e26d35-431b-4352-a37c-8eb00b31b7a5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:50.189581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 321/643] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:50.190464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6986d1a-e725-470c-9cf4-23ebd859099d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:50.226253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc349bb-e070-48d1-8663-0422eb7159a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:50.226967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc349bb-e070-48d1-8663-0422eb7159a3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.227246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc349bb-e070-48d1-8663-0422eb7159a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.227517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc349bb-e070-48d1-8663-0422eb7159a3 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.228360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc349bb-e070-48d1-8663-0422eb7159a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:50.228667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc349bb-e070-48d1-8663-0422eb7159a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:50.230102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc349bb-e070-48d1-8663-0422eb7159a3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:50.247505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 323/644] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jun 29 08:22:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:50.248331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc349bb-e070-48d1-8663-0422eb7159a3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:50.309529 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6986d1a-e725-470c-9cf4-23ebd859099d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:50.345758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270c98e-b517-47b0-a47e-223ad5cb94bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:50.346867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270c98e-b517-47b0-a47e-223ad5cb94bf None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.347279 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270c98e-b517-47b0-a47e-223ad5cb94bf None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.347569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270c98e-b517-47b0-a47e-223ad5cb94bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.353635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:50.355474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270c98e-b517-47b0-a47e-223ad5cb94bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:50.356988 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270c98e-b517-47b0-a47e-223ad5cb94bf None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:50.368008 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc349bb-e070-48d1-8663-0422eb7159a3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5270c98e-b517-47b0-a47e-223ad5cb94bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:50.372740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:50.373703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:50.374821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:50.374821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:50.374821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:50.374821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:50.374821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:50.374821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:50.374821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:50.374821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:50.379029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 322/645] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:50.380422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6700eec-c5f4-4c20-9b42-b5976e03022b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.380711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6700eec-c5f4-4c20-9b42-b5976e03022b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.381016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6700eec-c5f4-4c20-9b42-b5976e03022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.398296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a6700eec-c5f4-4c20-9b42-b5976e03022b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:50.399540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a6700eec-c5f4-4c20-9b42-b5976e03022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:50.404589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194a1a5-0294-4283-9900-6ced224fcece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:50.405390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194a1a5-0294-4283-9900-6ced224fcece None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.405671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194a1a5-0294-4283-9900-6ced224fcece None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.405943 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194a1a5-0294-4283-9900-6ced224fcece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.416727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194a1a5-0294-4283-9900-6ced224fcece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:50.418182 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194a1a5-0294-4283-9900-6ced224fcece None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:50.421614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-1194a1a5-0294-4283-9900-6ced224fcece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:50.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:50.437586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:50.443354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:50.443354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:50.443354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:50.443354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:50.443354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 324/646] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:50.443354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194a1a5-0294-4283-9900-6ced224fcece None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:50.481970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 323/647] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:50.482852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6700eec-c5f4-4c20-9b42-b5976e03022b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:50.551109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194a1a5-0294-4283-9900-6ced224fcece None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:50.587301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca3474-76e1-48bb-9370-960e721d21ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:50.587961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca3474-76e1-48bb-9370-960e721d21ba None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.588246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca3474-76e1-48bb-9370-960e721d21ba None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.588873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca3474-76e1-48bb-9370-960e721d21ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.592112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6700eec-c5f4-4c20-9b42-b5976e03022b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:50.600169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:50.629865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc78bd1-35ff-4eaa-8202-38879f50491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:50.630534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78bd1-35ff-4eaa-8202-38879f50491f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.630848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78bd1-35ff-4eaa-8202-38879f50491f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.631125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc78bd1-35ff-4eaa-8202-38879f50491f None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.631922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc78bd1-35ff-4eaa-8202-38879f50491f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:50.632233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc78bd1-35ff-4eaa-8202-38879f50491f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:50.633687 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc78bd1-35ff-4eaa-8202-38879f50491f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:50.646138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 324/648] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:50.647307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc78bd1-35ff-4eaa-8202-38879f50491f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:50.662627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca3474-76e1-48bb-9370-960e721d21ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:50.664112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca3474-76e1-48bb-9370-960e721d21ba None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:50.670916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:50.764355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc78bd1-35ff-4eaa-8202-38879f50491f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:50.772123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 325/649] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:50.773040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca3474-76e1-48bb-9370-960e721d21ba None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:50.805876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071694a-20c3-4ab0-a228-83ed583dc006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:50.806543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071694a-20c3-4ab0-a228-83ed583dc006 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.811881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071694a-20c3-4ab0-a228-83ed583dc006 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.811881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071694a-20c3-4ab0-a228-83ed583dc006 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.811881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071694a-20c3-4ab0-a228-83ed583dc006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:50.811881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071694a-20c3-4ab0-a228-83ed583dc006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:50.811881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071694a-20c3-4ab0-a228-83ed583dc006 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:50.821775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:50.831907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 325/650] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:50.831907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071694a-20c3-4ab0-a228-83ed583dc006 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:50.890188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca3474-76e1-48bb-9370-960e721d21ba None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:50.907254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:50.924808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724d70e-8319-4f40-8ee5-56fdeb839c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:50.925611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724d70e-8319-4f40-8ee5-56fdeb839c55 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.925895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724d70e-8319-4f40-8ee5-56fdeb839c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.926156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724d70e-8319-4f40-8ee5-56fdeb839c55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.932519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071694a-20c3-4ab0-a228-83ed583dc006 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:50.936629 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724d70e-8319-4f40-8ee5-56fdeb839c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-8724d70e-8319-4f40-8ee5-56fdeb839c55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:50.944461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:50.945451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:50.946346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:50.950594 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 326/651] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:50.951452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724d70e-8319-4f40-8ee5-56fdeb839c55 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:50.973409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4dbb1-7e00-449f-b112-d2f6501f5417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:50.974076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4dbb1-7e00-449f-b112-d2f6501f5417 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:50.974352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4dbb1-7e00-449f-b112-d2f6501f5417 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:50.974862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4dbb1-7e00-449f-b112-d2f6501f5417 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:50.975508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4dbb1-7e00-449f-b112-d2f6501f5417 None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=7bd500f020f143339c5a4fc9f0728ee4) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:50.975826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4dbb1-7e00-449f-b112-d2f6501f5417 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:50.977278 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4dbb1-7e00-449f-b112-d2f6501f5417 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:50.983566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 326/652] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:50 2022] GET /identity/v3/role_assignments?user.id=7bd500f020f143339c5a4fc9f0728ee4&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:50.987932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4dbb1-7e00-449f-b112-d2f6501f5417 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:51.060453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:51.063467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724d70e-8319-4f40-8ee5-56fdeb839c55 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:51.115390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e3987-0be4-4b24-8e39-e74f174bef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:51.116634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e3987-0be4-4b24-8e39-e74f174bef6c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:51.117139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e3987-0be4-4b24-8e39-e74f174bef6c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:51.117613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e3987-0be4-4b24-8e39-e74f174bef6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:51.132178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4dbb1-7e00-449f-b112-d2f6501f5417 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:51.141034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e3987-0be4-4b24-8e39-e74f174bef6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:51.143640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e3987-0be4-4b24-8e39-e74f174bef6c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-2d4e3987-0be4-4b24-8e39-e74f174bef6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:51.166953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:51.167901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:51.168771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:51.168771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:51.168771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:51.168771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:51.172308 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 327/653] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:51.173114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e3987-0be4-4b24-8e39-e74f174bef6c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:51.179837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca708c-0558-433f-aefa-b5d1499a55e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:51.180683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca708c-0558-433f-aefa-b5d1499a55e1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:51.180938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca708c-0558-433f-aefa-b5d1499a55e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:51.181214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca708c-0558-433f-aefa-b5d1499a55e1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:51.191748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca708c-0558-433f-aefa-b5d1499a55e1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:51.194116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca708c-0558-433f-aefa-b5d1499a55e1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:51.197277 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f0ca708c-0558-433f-aefa-b5d1499a55e1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:51.210949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:51.211913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:51.212850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:51.212850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:51.212850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:51.212850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:51.212850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:51.212850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:51.212850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:51.212850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:51.212850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 08:22:51.212850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:51.215493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 327/654] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:51.216497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca708c-0558-433f-aefa-b5d1499a55e1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:51.309284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e3987-0be4-4b24-8e39-e74f174bef6c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:51.345170 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144ed6d-950e-40a1-9ae2-a067f5f4500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:51.345412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca708c-0558-433f-aefa-b5d1499a55e1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:51.345850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144ed6d-950e-40a1-9ae2-a067f5f4500a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:51.346208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144ed6d-950e-40a1-9ae2-a067f5f4500a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:51.346505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144ed6d-950e-40a1-9ae2-a067f5f4500a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:51.347231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144ed6d-950e-40a1-9ae2-a067f5f4500a None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:51.347538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144ed6d-950e-40a1-9ae2-a067f5f4500a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:51.348944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144ed6d-950e-40a1-9ae2-a067f5f4500a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:51.357704 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 328/655] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:51.358507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144ed6d-950e-40a1-9ae2-a067f5f4500a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:51.375801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:51.389117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd19e8-d458-44e0-8c8c-d308171e0ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:51.389784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd19e8-d458-44e0-8c8c-d308171e0ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:51.390081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd19e8-d458-44e0-8c8c-d308171e0ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:51.390329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd19e8-d458-44e0-8c8c-d308171e0ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:51.391156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd19e8-d458-44e0-8c8c-d308171e0ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:51.391404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd19e8-d458-44e0-8c8c-d308171e0ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:51.392821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd19e8-d458-44e0-8c8c-d308171e0ad8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:51.405741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 328/656] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:51.407058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734223b-320f-4a7b-a284-aa19b930d374 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:51.407350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734223b-320f-4a7b-a284-aa19b930d374 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:51.407627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734223b-320f-4a7b-a284-aa19b930d374 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:51.409730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 329/657] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:51.410276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734223b-320f-4a7b-a284-aa19b930d374 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:51.476540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144ed6d-950e-40a1-9ae2-a067f5f4500a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:51.480109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:51.512317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aff3e7-ee45-47a5-b068-e0e39dffc27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:51.513037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aff3e7-ee45-47a5-b068-e0e39dffc27f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:51.513266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aff3e7-ee45-47a5-b068-e0e39dffc27f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:51.513538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aff3e7-ee45-47a5-b068-e0e39dffc27f None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:51.514376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aff3e7-ee45-47a5-b068-e0e39dffc27f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:51.514682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aff3e7-ee45-47a5-b068-e0e39dffc27f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:51.516142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aff3e7-ee45-47a5-b068-e0e39dffc27f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:51.527225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734223b-320f-4a7b-a284-aa19b930d374 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:51.535000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 329/658] 10.222.0.27 () {56 vars in 1137 bytes} [Wed Jun 29 08:22:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:51.536236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aff3e7-ee45-47a5-b068-e0e39dffc27f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:51.569598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176db9a-d860-48ca-86e0-cb0cf7c26ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:51.570923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176db9a-d860-48ca-86e0-cb0cf7c26ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:51.570923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176db9a-d860-48ca-86e0-cb0cf7c26ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:51.571109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176db9a-d860-48ca-86e0-cb0cf7c26ea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:51.586062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176db9a-d860-48ca-86e0-cb0cf7c26ea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:51.587911 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176db9a-d860-48ca-86e0-cb0cf7c26ea1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0176db9a-d860-48ca-86e0-cb0cf7c26ea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:51.610641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:51.611628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:51.613054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:51.613054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:51.613054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:51.613054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:51.613054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 330/659] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:51.613610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7dc0f-dd0c-405b-8e77-8fc61d0725eb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:51.613893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7dc0f-dd0c-405b-8e77-8fc61d0725eb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:51.614167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7dc0f-dd0c-405b-8e77-8fc61d0725eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:51.617637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:51.634564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-99f7dc0f-dd0c-405b-8e77-8fc61d0725eb None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:51.635785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-99f7dc0f-dd0c-405b-8e77-8fc61d0725eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:51.652061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aff3e7-ee45-47a5-b068-e0e39dffc27f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:51.687382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cac998-6827-491d-994c-2d6046e5679e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:51.688185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cac998-6827-491d-994c-2d6046e5679e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:51.688467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cac998-6827-491d-994c-2d6046e5679e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:51.688744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cac998-6827-491d-994c-2d6046e5679e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:51.696277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cac998-6827-491d-994c-2d6046e5679e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:51.697930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cac998-6827-491d-994c-2d6046e5679e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:51.714715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 331/660] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:51.714715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7dc0f-dd0c-405b-8e77-8fc61d0725eb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-76cac998-6827-491d-994c-2d6046e5679e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:51.714938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:51.715768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:51.716647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:51.716647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:51.716647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:51.716647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:51.716647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:51.716647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:51.716647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:51.716647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:51.716647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 330/661] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:51.716647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cac998-6827-491d-994c-2d6046e5679e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:51.762903 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:51.820800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7dc0f-dd0c-405b-8e77-8fc61d0725eb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:51.824847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cac998-6827-491d-994c-2d6046e5679e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:51.856232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b343-f461-4037-b651-48a7a32b52c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:51.856891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b343-f461-4037-b651-48a7a32b52c3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:51.857170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b343-f461-4037-b651-48a7a32b52c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:51.857443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932b343-f461-4037-b651-48a7a32b52c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:51.857917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:51.864048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0e641-0b27-443c-9057-8cd8d76210c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:51.865134 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0e641-0b27-443c-9057-8cd8d76210c3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:51.865419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0e641-0b27-443c-9057-8cd8d76210c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:51.865696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0e641-0b27-443c-9057-8cd8d76210c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:51.866521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0e641-0b27-443c-9057-8cd8d76210c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:51.866772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0e641-0b27-443c-9057-8cd8d76210c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:51.868152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0e641-0b27-443c-9057-8cd8d76210c3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:51.880397 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 331/662] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:51.885325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0e641-0b27-443c-9057-8cd8d76210c3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:51.927912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932b343-f461-4037-b651-48a7a32b52c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:51.929284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932b343-f461-4037-b651-48a7a32b52c3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:51.975386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:51.989336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0e641-0b27-443c-9057-8cd8d76210c3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:52.026461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad38af3-b783-4a49-8b2a-d7b8746d1e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:52.027410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad38af3-b783-4a49-8b2a-d7b8746d1e20 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:52.027695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad38af3-b783-4a49-8b2a-d7b8746d1e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:52.028115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad38af3-b783-4a49-8b2a-d7b8746d1e20 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:52.029216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad38af3-b783-4a49-8b2a-d7b8746d1e20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:52.029666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad38af3-b783-4a49-8b2a-d7b8746d1e20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:52.031422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad38af3-b783-4a49-8b2a-d7b8746d1e20 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:52.042422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 332/663] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:52.042422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b343-f461-4037-b651-48a7a32b52c3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:52.044276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 332/664] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:52.045160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad38af3-b783-4a49-8b2a-d7b8746d1e20 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:52.102670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:52.149746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932b343-f461-4037-b651-48a7a32b52c3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:52.152480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad38af3-b783-4a49-8b2a-d7b8746d1e20 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:52.184910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082db72-8b1b-4f2f-8ebd-efcc3d5cff42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:52.185646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082db72-8b1b-4f2f-8ebd-efcc3d5cff42 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:52.185891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082db72-8b1b-4f2f-8ebd-efcc3d5cff42 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:52.186155 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082db72-8b1b-4f2f-8ebd-efcc3d5cff42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:52.188457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49853ba7-cd69-43a9-80ed-07bc0a6a5a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:52.189257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49853ba7-cd69-43a9-80ed-07bc0a6a5a8a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:52.189536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49853ba7-cd69-43a9-80ed-07bc0a6a5a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:52.190051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49853ba7-cd69-43a9-80ed-07bc0a6a5a8a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:52.196776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6082db72-8b1b-4f2f-8ebd-efcc3d5cff42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:52.199102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49853ba7-cd69-43a9-80ed-07bc0a6a5a8a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:52.200579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49853ba7-cd69-43a9-80ed-07bc0a6a5a8a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6082db72-8b1b-4f2f-8ebd-efcc3d5cff42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:52.204658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:52.205678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 333/665] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:52.211409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082db72-8b1b-4f2f-8ebd-efcc3d5cff42 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:52.214259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-49853ba7-cd69-43a9-80ed-07bc0a6a5a8a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:52.222881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:52.223883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:52.224823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:52.224823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:52.224823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:52.224823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:52.224823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:52.224823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:52.224823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:52.224823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:52.224823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 08:22:52.224823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:52.224823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 333/666] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:52.225480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49853ba7-cd69-43a9-80ed-07bc0a6a5a8a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:52.322782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6082db72-8b1b-4f2f-8ebd-efcc3d5cff42 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:52.332883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49853ba7-cd69-43a9-80ed-07bc0a6a5a8a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:52.347914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:52.361187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15e467-7e8d-4a7e-99eb-a1de1c3b3d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:52.362061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e467-7e8d-4a7e-99eb-a1de1c3b3d1c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:52.362376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e467-7e8d-4a7e-99eb-a1de1c3b3d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:52.362614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e467-7e8d-4a7e-99eb-a1de1c3b3d1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:52.370776 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d107426-a1f1-44b9-ba99-fedc7d98fab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:52.371425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d107426-a1f1-44b9-ba99-fedc7d98fab9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:52.371730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d107426-a1f1-44b9-ba99-fedc7d98fab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:52.371977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d107426-a1f1-44b9-ba99-fedc7d98fab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:52.372817 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d107426-a1f1-44b9-ba99-fedc7d98fab9 None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=fd09453b5db645d7b03522f91effb8d4) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:52.373132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d107426-a1f1-44b9-ba99-fedc7d98fab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:52.374204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15e467-7e8d-4a7e-99eb-a1de1c3b3d1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:52.374546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d107426-a1f1-44b9-ba99-fedc7d98fab9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:52.376074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15e467-7e8d-4a7e-99eb-a1de1c3b3d1c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:52.380509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 334/667] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:52 2022] GET /identity/v3/role_assignments?user.id=fd09453b5db645d7b03522f91effb8d4&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:52.381281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d107426-a1f1-44b9-ba99-fedc7d98fab9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0e15e467-7e8d-4a7e-99eb-a1de1c3b3d1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:52.404582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:52.405580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:52.406472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:52.406472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:52.406472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:52.406472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:52.408979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 334/668] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:52.410108 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15e467-7e8d-4a7e-99eb-a1de1c3b3d1c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:52.480854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:52.520774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d107426-a1f1-44b9-ba99-fedc7d98fab9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:52.535173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15e467-7e8d-4a7e-99eb-a1de1c3b3d1c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:52.567793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df1f94-05b7-436f-9104-ec505fa9742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:52.568571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df1f94-05b7-436f-9104-ec505fa9742a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:52.568924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df1f94-05b7-436f-9104-ec505fa9742a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:52.569278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df1f94-05b7-436f-9104-ec505fa9742a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:52.570043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df1f94-05b7-436f-9104-ec505fa9742a None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:52.570361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df1f94-05b7-436f-9104-ec505fa9742a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:52.571782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df1f94-05b7-436f-9104-ec505fa9742a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:52.573484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beeb9e-b092-4ee1-a7a9-1eda4ab71dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:52.574157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beeb9e-b092-4ee1-a7a9-1eda4ab71dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:52.574438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beeb9e-b092-4ee1-a7a9-1eda4ab71dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:52.574738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beeb9e-b092-4ee1-a7a9-1eda4ab71dc1 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:52.575582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1beeb9e-b092-4ee1-a7a9-1eda4ab71dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:52.575890 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1beeb9e-b092-4ee1-a7a9-1eda4ab71dc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:52.577311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1beeb9e-b092-4ee1-a7a9-1eda4ab71dc1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:52.582719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 335/669] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:52.582719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df1f94-05b7-436f-9104-ec505fa9742a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:52.606260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 335/670] 10.222.0.27 () {56 vars in 1139 bytes} [Wed Jun 29 08:22:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:52.610880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beeb9e-b092-4ee1-a7a9-1eda4ab71dc1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:52.638874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:52.710098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df1f94-05b7-436f-9104-ec505fa9742a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:52.736395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beeb9e-b092-4ee1-a7a9-1eda4ab71dc1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:52.745034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387eb3-45d5-4d73-bea4-ad8bd398c79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:52.745708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387eb3-45d5-4d73-bea4-ad8bd398c79e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:52.745983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387eb3-45d5-4d73-bea4-ad8bd398c79e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:52.746256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387eb3-45d5-4d73-bea4-ad8bd398c79e None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:52.747282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00387eb3-45d5-4d73-bea4-ad8bd398c79e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:52.747404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00387eb3-45d5-4d73-bea4-ad8bd398c79e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:52.748669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00387eb3-45d5-4d73-bea4-ad8bd398c79e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:52.754584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:52.761446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 336/671] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:52.766023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387eb3-45d5-4d73-bea4-ad8bd398c79e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:52.786187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51385636-f0e4-4159-ab80-8ec992fc127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:52.787859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51385636-f0e4-4159-ab80-8ec992fc127e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:52.788342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51385636-f0e4-4159-ab80-8ec992fc127e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:52.789090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51385636-f0e4-4159-ab80-8ec992fc127e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:52.803110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51385636-f0e4-4159-ab80-8ec992fc127e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:52.804863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51385636-f0e4-4159-ab80-8ec992fc127e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-51385636-f0e4-4159-ab80-8ec992fc127e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:52.827497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:52.828442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:52.829297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:52.829297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:52.829297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:52.829297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:52.832262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 336/672] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:52.833315 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51385636-f0e4-4159-ab80-8ec992fc127e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:52.874529 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:52.880463 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387eb3-45d5-4d73-bea4-ad8bd398c79e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:52.919693 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d48ceb-3738-4584-a5e6-bd95a66ff9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:52.920527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d48ceb-3738-4584-a5e6-bd95a66ff9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:52.920923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d48ceb-3738-4584-a5e6-bd95a66ff9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:52.921291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d48ceb-3738-4584-a5e6-bd95a66ff9d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:52.965582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51385636-f0e4-4159-ab80-8ec992fc127e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:53.002249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a4b60-b69c-4db0-9465-4046822ddd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:53.003544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a4b60-b69c-4db0-9465-4046822ddd0a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.003741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a4b60-b69c-4db0-9465-4046822ddd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.004109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a4b60-b69c-4db0-9465-4046822ddd0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:53.006288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d48ceb-3738-4584-a5e6-bd95a66ff9d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:53.008251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d48ceb-3738-4584-a5e6-bd95a66ff9d0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:53.011996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a4b60-b69c-4db0-9465-4046822ddd0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:53.013540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a4b60-b69c-4db0-9465-4046822ddd0a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c14a4b60-b69c-4db0-9465-4046822ddd0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:53.028818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:53.029927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:53.031015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:53.031015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:53.031015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:53.031015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:53.031015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:53.031015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:53.031015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:53.031015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:53.037185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 337/673] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:53.037185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a4b60-b69c-4db0-9465-4046822ddd0a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:53.067149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:53.137680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:53.138405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a4b60-b69c-4db0-9465-4046822ddd0a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:53.144017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 337/674] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:53.145193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d48ceb-3738-4584-a5e6-bd95a66ff9d0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:53.183724 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe82b2d-b2ca-453b-85fe-8b85aa4ebffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:53.184392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe82b2d-b2ca-453b-85fe-8b85aa4ebffb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.184664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe82b2d-b2ca-453b-85fe-8b85aa4ebffb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.184936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe82b2d-b2ca-453b-85fe-8b85aa4ebffb None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:53.185665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe82b2d-b2ca-453b-85fe-8b85aa4ebffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:53.185976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe82b2d-b2ca-453b-85fe-8b85aa4ebffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:53.187396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe82b2d-b2ca-453b-85fe-8b85aa4ebffb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:53.203458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 338/675] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:53.203458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab878ba-e4cd-4d98-8a7a-53586ef28aff None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.203458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab878ba-e4cd-4d98-8a7a-53586ef28aff None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.203458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab878ba-e4cd-4d98-8a7a-53586ef28aff None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:53.203458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 339/676] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:53.203458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab878ba-e4cd-4d98-8a7a-53586ef28aff None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:53.260539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d48ceb-3738-4584-a5e6-bd95a66ff9d0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:53.297135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b8aa7-25c9-4cb3-9cdc-bfce4cfdeea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:53.297381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:53.297802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b8aa7-25c9-4cb3-9cdc-bfce4cfdeea5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.298084 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b8aa7-25c9-4cb3-9cdc-bfce4cfdeea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.298357 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b8aa7-25c9-4cb3-9cdc-bfce4cfdeea5 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:53.299332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b8aa7-25c9-4cb3-9cdc-bfce4cfdeea5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:53.299889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b8aa7-25c9-4cb3-9cdc-bfce4cfdeea5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:53.301373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b8aa7-25c9-4cb3-9cdc-bfce4cfdeea5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:53.313775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 338/677] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:53.315120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b8aa7-25c9-4cb3-9cdc-bfce4cfdeea5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:53.316369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab878ba-e4cd-4d98-8a7a-53586ef28aff None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:53.355330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d76237-f11b-4773-b0f2-c35930e018c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:53.356159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d76237-f11b-4773-b0f2-c35930e018c3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.356444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d76237-f11b-4773-b0f2-c35930e018c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.356716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d76237-f11b-4773-b0f2-c35930e018c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:53.367232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d76237-f11b-4773-b0f2-c35930e018c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:53.368673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d76237-f11b-4773-b0f2-c35930e018c3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:53.376591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-92d76237-f11b-4773-b0f2-c35930e018c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:53.386630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:53.387627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:53.388524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:53.388524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:53.388524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:53.388524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:53.388524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 340/678] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:53.389323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d7a7-b269-4faa-893b-55ed8abbe01f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.389432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d7a7-b269-4faa-893b-55ed8abbe01f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.389645 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9d7a7-b269-4faa-893b-55ed8abbe01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:53.406964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-92f9d7a7-b269-4faa-893b-55ed8abbe01f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:53.408161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-92f9d7a7-b269-4faa-893b-55ed8abbe01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:53.425584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b8aa7-25c9-4cb3-9cdc-bfce4cfdeea5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:53.470827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4ed48-a91d-4779-9629-1c33c1240ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:53.472420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ed48-a91d-4779-9629-1c33c1240ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.472704 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ed48-a91d-4779-9629-1c33c1240ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.472981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4ed48-a91d-4779-9629-1c33c1240ed9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:53.486827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 341/679] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:53.486827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9d7a7-b269-4faa-893b-55ed8abbe01f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:53.488818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4ed48-a91d-4779-9629-1c33c1240ed9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:53.491344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4ed48-a91d-4779-9629-1c33c1240ed9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-31d4ed48-a91d-4779-9629-1c33c1240ed9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:53.507438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:53.508613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:53.509732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:53.509732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:53.509732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:53.509732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:53.509732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:53.509732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:53.509732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:53.509732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:53.509732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:22:53.509732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:53.513228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 339/680] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:22:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:53.514074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4ed48-a91d-4779-9629-1c33c1240ed9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:53.555706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:53.596840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9d7a7-b269-4faa-893b-55ed8abbe01f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:53.621850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4ed48-a91d-4779-9629-1c33c1240ed9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:53.633962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd38928-8ccd-4d7b-9527-1be1c65b7848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:53.634626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38928-8ccd-4d7b-9527-1be1c65b7848 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.635200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38928-8ccd-4d7b-9527-1be1c65b7848 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.635480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd38928-8ccd-4d7b-9527-1be1c65b7848 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:53.636790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd38928-8ccd-4d7b-9527-1be1c65b7848 None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=b60ca33edf5a47fdab6e39cc98638c6d) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:53.637365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd38928-8ccd-4d7b-9527-1be1c65b7848 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:53.639662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:53.640362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd38928-8ccd-4d7b-9527-1be1c65b7848 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:53.652623 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 342/681] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:53 2022] GET /identity/v3/role_assignments?user.id=b60ca33edf5a47fdab6e39cc98638c6d&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:53.653884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd38928-8ccd-4d7b-9527-1be1c65b7848 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:53.667991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2408fbb-d88b-481b-a4eb-bed535d76a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:53.669178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2408fbb-d88b-481b-a4eb-bed535d76a5c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.669659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2408fbb-d88b-481b-a4eb-bed535d76a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.670133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2408fbb-d88b-481b-a4eb-bed535d76a5c None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:53.671333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2408fbb-d88b-481b-a4eb-bed535d76a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:53.671916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2408fbb-d88b-481b-a4eb-bed535d76a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:53.673621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2408fbb-d88b-481b-a4eb-bed535d76a5c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:53.693217 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 340/682] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:53.694650 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2408fbb-d88b-481b-a4eb-bed535d76a5c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:53.767511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd38928-8ccd-4d7b-9527-1be1c65b7848 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:53.798176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:53.804554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27918ea4-2c06-4541-a44c-e097df010cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:53.805674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27918ea4-2c06-4541-a44c-e097df010cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.806157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27918ea4-2c06-4541-a44c-e097df010cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.806668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27918ea4-2c06-4541-a44c-e097df010cf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:53.806910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2408fbb-d88b-481b-a4eb-bed535d76a5c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:53.818913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27918ea4-2c06-4541-a44c-e097df010cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-27918ea4-2c06-4541-a44c-e097df010cf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:53.826913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:53.827944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:53.828783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 343/683] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:22:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:53.829477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27918ea4-2c06-4541-a44c-e097df010cf0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:53.855230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b35ccd-7ea0-4d6c-b496-713e0afb73c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:53.855467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b35ccd-7ea0-4d6c-b496-713e0afb73c2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.855696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b35ccd-7ea0-4d6c-b496-713e0afb73c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.855854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b35ccd-7ea0-4d6c-b496-713e0afb73c2 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:53.856858 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b35ccd-7ea0-4d6c-b496-713e0afb73c2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:53.857172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b35ccd-7ea0-4d6c-b496-713e0afb73c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:53.858644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b35ccd-7ea0-4d6c-b496-713e0afb73c2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:53.878407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 341/684] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jun 29 08:22:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:53.880230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b35ccd-7ea0-4d6c-b496-713e0afb73c2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:53.931828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:53.955430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27918ea4-2c06-4541-a44c-e097df010cf0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:53.993508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9f2fd-4797-4b62-9e84-4bce38527618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:53.994316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9f2fd-4797-4b62-9e84-4bce38527618 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:53.994602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9f2fd-4797-4b62-9e84-4bce38527618 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:53.994994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9f2fd-4797-4b62-9e84-4bce38527618 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.002568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9f2fd-4797-4b62-9e84-4bce38527618 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:54.004185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea9f2fd-4797-4b62-9e84-4bce38527618 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-fea9f2fd-4797-4b62-9e84-4bce38527618 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:54.019535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:54.020465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:54.021955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:54.021955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:54.021955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:54.021955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:54.021955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:54.021955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:54.021955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:54.021955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:54.021955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 344/685] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:54.021955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9f2fd-4797-4b62-9e84-4bce38527618 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:54.027394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b35ccd-7ea0-4d6c-b496-713e0afb73c2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:54.078850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da45698-f98b-42ab-ba95-6539691b1c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:54.079782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da45698-f98b-42ab-ba95-6539691b1c4f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:54.080067 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da45698-f98b-42ab-ba95-6539691b1c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:54.080488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da45698-f98b-42ab-ba95-6539691b1c4f None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.081177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:54.082514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da45698-f98b-42ab-ba95-6539691b1c4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:54.083034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da45698-f98b-42ab-ba95-6539691b1c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:54.084981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da45698-f98b-42ab-ba95-6539691b1c4f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:54.098496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 342/686] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:54.099385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da45698-f98b-42ab-ba95-6539691b1c4f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:54.137400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9f2fd-4797-4b62-9e84-4bce38527618 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:54.174079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e777e-32c2-4a4b-b06b-2b97e21774ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:54.174956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e777e-32c2-4a4b-b06b-2b97e21774ea None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:54.175367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e777e-32c2-4a4b-b06b-2b97e21774ea None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:54.175737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e777e-32c2-4a4b-b06b-2b97e21774ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.200167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:54.204333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da45698-f98b-42ab-ba95-6539691b1c4f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:54.240627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81e5be-e7b5-4800-b333-5955c47f42ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:54.241582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81e5be-e7b5-4800-b333-5955c47f42ee None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:54.242001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81e5be-e7b5-4800-b333-5955c47f42ee None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:54.242410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81e5be-e7b5-4800-b333-5955c47f42ee None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.243390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81e5be-e7b5-4800-b333-5955c47f42ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:54.243850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81e5be-e7b5-4800-b333-5955c47f42ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:54.245556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81e5be-e7b5-4800-b333-5955c47f42ee None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:54.250986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e777e-32c2-4a4b-b06b-2b97e21774ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:54.252749 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e777e-32c2-4a4b-b06b-2b97e21774ea None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:54.263052 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 343/687] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:54.264243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81e5be-e7b5-4800-b333-5955c47f42ee None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:54.304777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:54.364502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 345/688] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:54.364502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e777e-32c2-4a4b-b06b-2b97e21774ea None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:54.372311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81e5be-e7b5-4800-b333-5955c47f42ee None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:54.408077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c0ad0-c35c-49c5-aea1-75eaddc5e3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:54.409875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c0ad0-c35c-49c5-aea1-75eaddc5e3bf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:54.410371 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c0ad0-c35c-49c5-aea1-75eaddc5e3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:54.410868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c0ad0-c35c-49c5-aea1-75eaddc5e3bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.421693 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c0ad0-c35c-49c5-aea1-75eaddc5e3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:54.423415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c0ad0-c35c-49c5-aea1-75eaddc5e3bf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-423c0ad0-c35c-49c5-aea1-75eaddc5e3bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:54.441187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:54.442115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:54.443016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:54.443016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:54.443016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:54.443016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:54.445325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 344/689] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:54.446475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c0ad0-c35c-49c5-aea1-75eaddc5e3bf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:54.463091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:54.466907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e777e-32c2-4a4b-b06b-2b97e21774ea None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:54.503672 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfc52f-5a7d-4591-8b0d-dc975d34f026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:54.504331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfc52f-5a7d-4591-8b0d-dc975d34f026 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:54.504618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfc52f-5a7d-4591-8b0d-dc975d34f026 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:54.504896 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bfc52f-5a7d-4591-8b0d-dc975d34f026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.505741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bfc52f-5a7d-4591-8b0d-dc975d34f026 None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=49a1439029fa4b719bc114c3f59e0050) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:54.506056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bfc52f-5a7d-4591-8b0d-dc975d34f026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:54.507528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bfc52f-5a7d-4591-8b0d-dc975d34f026 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:54.522721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 346/690] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:54 2022] GET /identity/v3/role_assignments?user.id=49a1439029fa4b719bc114c3f59e0050&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:54.522721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfc52f-5a7d-4591-8b0d-dc975d34f026 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:54.532372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:54.602719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c0ad0-c35c-49c5-aea1-75eaddc5e3bf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:54.635037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bfc52f-5a7d-4591-8b0d-dc975d34f026 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:54.643580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98da335-8b6f-44aa-a170-8dea9e2b3944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:54.644771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98da335-8b6f-44aa-a170-8dea9e2b3944 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:54.645212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98da335-8b6f-44aa-a170-8dea9e2b3944 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:54.645640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98da335-8b6f-44aa-a170-8dea9e2b3944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.658039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98da335-8b6f-44aa-a170-8dea9e2b3944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:54.660080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98da335-8b6f-44aa-a170-8dea9e2b3944 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:54.674028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87d108-31da-4386-9012-6d4d593d640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:54.674977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87d108-31da-4386-9012-6d4d593d640b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:54.675318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87d108-31da-4386-9012-6d4d593d640b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:54.675613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b87d108-31da-4386-9012-6d4d593d640b None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.676362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87d108-31da-4386-9012-6d4d593d640b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:54.676688 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87d108-31da-4386-9012-6d4d593d640b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:54.678267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b87d108-31da-4386-9012-6d4d593d640b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f98da335-8b6f-44aa-a170-8dea9e2b3944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:54.681153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:54.682129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:54.683035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:54.683035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:54.683035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:54.683035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:54.685432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 345/691] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:54.691758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdb40d-94df-41d1-b300-2cc79292b4a7 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:54.692045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 347/692] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:54.692513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdb40d-94df-41d1-b300-2cc79292b4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:54.692907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87d108-31da-4386-9012-6d4d593d640b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:54.693207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdb40d-94df-41d1-b300-2cc79292b4a7 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.695145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 346/693] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:54.701701 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8d476-cede-42c1-b077-de70b98b243c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:54.701981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8d476-cede-42c1-b077-de70b98b243c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:54.704282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8d476-cede-42c1-b077-de70b98b243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.721638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8ca8d476-cede-42c1-b077-de70b98b243c None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:54.723273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8ca8d476-cede-42c1-b077-de70b98b243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:54.781436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:54.790152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:54.802683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 347/694] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:54.803674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8d476-cede-42c1-b077-de70b98b243c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:54.816609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b87d108-31da-4386-9012-6d4d593d640b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:54.859625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01b071-3a2e-4f82-aca6-e41a425b2fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:54.860550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01b071-3a2e-4f82-aca6-e41a425b2fae None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:54.860829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01b071-3a2e-4f82-aca6-e41a425b2fae None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:54.861101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01b071-3a2e-4f82-aca6-e41a425b2fae None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.862259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01b071-3a2e-4f82-aca6-e41a425b2fae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:54.862568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01b071-3a2e-4f82-aca6-e41a425b2fae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:54.864045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b01b071-3a2e-4f82-aca6-e41a425b2fae None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:54.879523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 348/695] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:54.880512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01b071-3a2e-4f82-aca6-e41a425b2fae None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:54.937650 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8d476-cede-42c1-b077-de70b98b243c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:54.982271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721602b6-d2b0-4e9b-8d3e-0025538bf6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:54.983189 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721602b6-d2b0-4e9b-8d3e-0025538bf6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:54.983407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721602b6-d2b0-4e9b-8d3e-0025538bf6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:54.983682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721602b6-d2b0-4e9b-8d3e-0025538bf6b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:54.991332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721602b6-d2b0-4e9b-8d3e-0025538bf6b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:54.992000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01b071-3a2e-4f82-aca6-e41a425b2fae None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:54.992849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721602b6-d2b0-4e9b-8d3e-0025538bf6b4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-721602b6-d2b0-4e9b-8d3e-0025538bf6b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:55.008405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:55.009410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:55.010343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:55.010343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:55.010343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:55.010343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:55.010343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:55.010343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:55.010343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:55.010343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:55.010343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:22:55.010343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:55.014336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 348/696] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jun 29 08:22:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:55.015224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721602b6-d2b0-4e9b-8d3e-0025538bf6b4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:55.037224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830f3cc-6a72-45d5-8b67-d52c2761c4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:55.038916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830f3cc-6a72-45d5-8b67-d52c2761c4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:55.039423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830f3cc-6a72-45d5-8b67-d52c2761c4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:55.040138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a830f3cc-6a72-45d5-8b67-d52c2761c4d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:55.042941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:55.056680 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830f3cc-6a72-45d5-8b67-d52c2761c4d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:55.058412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a830f3cc-6a72-45d5-8b67-d52c2761c4d0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:55.074548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-a830f3cc-6a72-45d5-8b67-d52c2761c4d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:55.083537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:55.084422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:55.085342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:55.085342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:55.085342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:55.085342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:55.087028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 349/697] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:55.088032 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830f3cc-6a72-45d5-8b67-d52c2761c4d0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:55.135235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721602b6-d2b0-4e9b-8d3e-0025538bf6b4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:55.178466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00a818-9355-4751-b7d3-27d1bd8d72eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:55.179319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00a818-9355-4751-b7d3-27d1bd8d72eb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:55.179647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00a818-9355-4751-b7d3-27d1bd8d72eb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:55.179922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00a818-9355-4751-b7d3-27d1bd8d72eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:55.189426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00a818-9355-4751-b7d3-27d1bd8d72eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:55.191226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00a818-9355-4751-b7d3-27d1bd8d72eb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:55.207545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a830f3cc-6a72-45d5-8b67-d52c2761c4d0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-da00a818-9355-4751-b7d3-27d1bd8d72eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:55.209641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:55.210550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:55.211500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:55.211500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:55.211500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:55.211500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:55.211500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:55.211500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:55.211500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:55.211500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:55.211952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 349/698] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:55.214734 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00a818-9355-4751-b7d3-27d1bd8d72eb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:55.254784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf69415-1524-4d6a-83d2-0f10e087d197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:55.255418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf69415-1524-4d6a-83d2-0f10e087d197 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:55.255697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf69415-1524-4d6a-83d2-0f10e087d197 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:55.255967 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf69415-1524-4d6a-83d2-0f10e087d197 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:55.256812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf69415-1524-4d6a-83d2-0f10e087d197 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:55.257117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf69415-1524-4d6a-83d2-0f10e087d197 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:55.259092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf69415-1524-4d6a-83d2-0f10e087d197 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:55.282322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 350/699] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jun 29 08:22:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:55.284379 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf69415-1524-4d6a-83d2-0f10e087d197 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:55.334247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:55.341452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00a818-9355-4751-b7d3-27d1bd8d72eb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:55.357955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:55.384747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbadb1-e854-49fe-994d-6c5f6ff1e8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:55.385478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbadb1-e854-49fe-994d-6c5f6ff1e8fb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:55.390746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbadb1-e854-49fe-994d-6c5f6ff1e8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:55.390746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effbadb1-e854-49fe-994d-6c5f6ff1e8fb None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:55.390746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbadb1-e854-49fe-994d-6c5f6ff1e8fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:55.390746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbadb1-e854-49fe-994d-6c5f6ff1e8fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:55.390746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effbadb1-e854-49fe-994d-6c5f6ff1e8fb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:55.399448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf69415-1524-4d6a-83d2-0f10e087d197 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:55.406986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 350/700] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:55.408116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbadb1-e854-49fe-994d-6c5f6ff1e8fb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:55.441302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46922956-174b-4e65-9b18-0500b569d354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:55.441970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46922956-174b-4e65-9b18-0500b569d354 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:55.446716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46922956-174b-4e65-9b18-0500b569d354 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:55.446716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46922956-174b-4e65-9b18-0500b569d354 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:55.446716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46922956-174b-4e65-9b18-0500b569d354 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:55.446716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46922956-174b-4e65-9b18-0500b569d354 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:55.446716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46922956-174b-4e65-9b18-0500b569d354 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:55.467511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 351/701] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:55.469019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46922956-174b-4e65-9b18-0500b569d354 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:55.535148 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effbadb1-e854-49fe-994d-6c5f6ff1e8fb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:55.574271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46922956-174b-4e65-9b18-0500b569d354 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:55.577012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:55.577171 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328a480-ec70-4f10-a417-d5c747ff9d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:55.577924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328a480-ec70-4f10-a417-d5c747ff9d65 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:55.578152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328a480-ec70-4f10-a417-d5c747ff9d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:55.578350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328a480-ec70-4f10-a417-d5c747ff9d65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:55.613012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2138e5-6cdd-4a59-bc6f-af88af24d07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:55.614037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2138e5-6cdd-4a59-bc6f-af88af24d07e None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:55.614527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2138e5-6cdd-4a59-bc6f-af88af24d07e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:55.615024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2138e5-6cdd-4a59-bc6f-af88af24d07e None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:55.615922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2138e5-6cdd-4a59-bc6f-af88af24d07e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:55.617610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2138e5-6cdd-4a59-bc6f-af88af24d07e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:55.629901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:55.651040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 352/702] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:22:55 2022] POST /identity/v3/users => generated 254 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:22:55.651040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2138e5-6cdd-4a59-bc6f-af88af24d07e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:55.677051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328a480-ec70-4f10-a417-d5c747ff9d65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:55.680295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328a480-ec70-4f10-a417-d5c747ff9d65 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:55.818679 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2138e5-6cdd-4a59-bc6f-af88af24d07e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:55.841976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 351/703] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:22:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:22:55.843490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae6e4d-2041-42b7-a33a-dab8dae35cee None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:55.843784 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae6e4d-2041-42b7-a33a-dab8dae35cee None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:55.844069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae6e4d-2041-42b7-a33a-dab8dae35cee None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:55.845773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 352/704] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:55.847761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae6e4d-2041-42b7-a33a-dab8dae35cee None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:55.862708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c313c94-3502-48a1-9841-96598200809d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:55.863413 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c313c94-3502-48a1-9841-96598200809d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:55.863697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c313c94-3502-48a1-9841-96598200809d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:55.863972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c313c94-3502-48a1-9841-96598200809d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:55.864926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c313c94-3502-48a1-9841-96598200809d None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=3b38411347754c058eb6640829078e1d) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:55.865242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c313c94-3502-48a1-9841-96598200809d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:55.866848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c313c94-3502-48a1-9841-96598200809d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:55.873752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 353/705] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:55 2022] GET /identity/v3/role_assignments?user.id=3b38411347754c058eb6640829078e1d&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:55.875071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec2242-4ad4-4797-a85c-8a2d4ff9900b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:55.875398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec2242-4ad4-4797-a85c-8a2d4ff9900b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:55.875683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec2242-4ad4-4797-a85c-8a2d4ff9900b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:55.894109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9fec2242-4ad4-4797-a85c-8a2d4ff9900b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:55.895373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9fec2242-4ad4-4797-a85c-8a2d4ff9900b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:55.918244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:55.932478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:55.976127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae6e4d-2041-42b7-a33a-dab8dae35cee None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:55.977098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 354/706] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:56.001338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec2242-4ad4-4797-a85c-8a2d4ff9900b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:56.018419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34165a-5ab6-43b4-8980-5784b98676bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:56.019419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34165a-5ab6-43b4-8980-5784b98676bc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:56.019709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34165a-5ab6-43b4-8980-5784b98676bc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:56.019998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34165a-5ab6-43b4-8980-5784b98676bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:56.033281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34165a-5ab6-43b4-8980-5784b98676bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:56.036617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34165a-5ab6-43b4-8980-5784b98676bc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-2d34165a-5ab6-43b4-8980-5784b98676bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:56.057936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:56.058928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:56.060042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:56.060042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:56.060042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:56.060042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:56.060042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 353/707] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:56.065405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34165a-5ab6-43b4-8980-5784b98676bc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:56.111681 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec2242-4ad4-4797-a85c-8a2d4ff9900b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:56.151065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32f3c5-af68-4c03-b930-1116e9baa270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:56.151885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32f3c5-af68-4c03-b930-1116e9baa270 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:56.152166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32f3c5-af68-4c03-b930-1116e9baa270 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:56.152437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32f3c5-af68-4c03-b930-1116e9baa270 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:56.153350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32f3c5-af68-4c03-b930-1116e9baa270 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:56.154755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32f3c5-af68-4c03-b930-1116e9baa270 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-cc32f3c5-af68-4c03-b930-1116e9baa270 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:56.171289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:56.172266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 08:22:56.173080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:56.174949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 355/708] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Jun 29 08:22:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:56.181340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32f3c5-af68-4c03-b930-1116e9baa270 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:56.204436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34165a-5ab6-43b4-8980-5784b98676bc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:56.205235 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:56.237090 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:56.246751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee958f-e0db-4492-bae5-85dd931176f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:56.247811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee958f-e0db-4492-bae5-85dd931176f5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:56.248291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee958f-e0db-4492-bae5-85dd931176f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:56.248755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee958f-e0db-4492-bae5-85dd931176f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:56.249708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee958f-e0db-4492-bae5-85dd931176f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:56.250209 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee958f-e0db-4492-bae5-85dd931176f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:56.251836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee958f-e0db-4492-bae5-85dd931176f5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:56.265962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 354/709] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:56.279112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee958f-e0db-4492-bae5-85dd931176f5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:56.324094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32f3c5-af68-4c03-b930-1116e9baa270 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:56.381884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9172f68-8604-4617-8f10-bdc0f5c18272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:56.382976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9172f68-8604-4617-8f10-bdc0f5c18272 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:56.383501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9172f68-8604-4617-8f10-bdc0f5c18272 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:56.383967 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9172f68-8604-4617-8f10-bdc0f5c18272 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:56.384892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9172f68-8604-4617-8f10-bdc0f5c18272 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:56.387069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9172f68-8604-4617-8f10-bdc0f5c18272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:56.388684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9172f68-8604-4617-8f10-bdc0f5c18272 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:56.404829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 356/710] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jun 29 08:22:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:56.416402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9172f68-8604-4617-8f10-bdc0f5c18272 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:56.489653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee958f-e0db-4492-bae5-85dd931176f5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:56.520211 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:56.538276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9172f68-8604-4617-8f10-bdc0f5c18272 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:56.552349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf75c43-87bb-4b7e-a5d9-e29f86865165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:56.553347 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf75c43-87bb-4b7e-a5d9-e29f86865165 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:56.553633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf75c43-87bb-4b7e-a5d9-e29f86865165 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:56.553908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf75c43-87bb-4b7e-a5d9-e29f86865165 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:56.571925 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf75c43-87bb-4b7e-a5d9-e29f86865165 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:56.573231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3db2b-708e-48fe-9b35-8f8d581374c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:56.573384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf75c43-87bb-4b7e-a5d9-e29f86865165 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:56.574001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3db2b-708e-48fe-9b35-8f8d581374c0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:56.574287 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3db2b-708e-48fe-9b35-8f8d581374c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:56.574560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3db2b-708e-48fe-9b35-8f8d581374c0 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:56.575309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3db2b-708e-48fe-9b35-8f8d581374c0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:56.575674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3db2b-708e-48fe-9b35-8f8d581374c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:56.577176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3db2b-708e-48fe-9b35-8f8d581374c0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:56.583847 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 357/711] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jun 29 08:22:56 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-0cf75c43-87bb-4b7e-a5d9-e29f86865165 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:22:56.595382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:56.596488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:56.597358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:22:56.597358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:22:56.597358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:22:56.597358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:22:56.597358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:22:56.597358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:22:56.597358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:22:56.597358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:56.598105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 355/712] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:56.604495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3db2b-708e-48fe-9b35-8f8d581374c0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:56.769593 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3db2b-708e-48fe-9b35-8f8d581374c0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:56.804589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53582d22-e9bf-4dfe-92c5-87ef4552b99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:56.805258 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582d22-e9bf-4dfe-92c5-87ef4552b99c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:56.805541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582d22-e9bf-4dfe-92c5-87ef4552b99c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:56.805815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53582d22-e9bf-4dfe-92c5-87ef4552b99c None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:56.806620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53582d22-e9bf-4dfe-92c5-87ef4552b99c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:56.806973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53582d22-e9bf-4dfe-92c5-87ef4552b99c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:56.808428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53582d22-e9bf-4dfe-92c5-87ef4552b99c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:56.823035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 358/713] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:22:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:56.828538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf75c43-87bb-4b7e-a5d9-e29f86865165 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:56.876462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20023304-4915-4f54-af92-96daa93907a8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:56.876807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20023304-4915-4f54-af92-96daa93907a8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:56.877016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20023304-4915-4f54-af92-96daa93907a8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:56.878648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 359/714] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:56.878775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:56.883862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b1987-1f90-4afa-b727-ab443bd9854e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:56.884168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b1987-1f90-4afa-b727-ab443bd9854e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:56.884415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b1987-1f90-4afa-b727-ab443bd9854e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:56.901737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:56.909391 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-329b1987-1f90-4afa-b727-ab443bd9854e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:56.910609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-329b1987-1f90-4afa-b727-ab443bd9854e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:56.975753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf75c43-87bb-4b7e-a5d9-e29f86865165 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:57.040891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d540ab-9dd4-4972-842e-b0984743adbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:57.042218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 360/715] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:57.042546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d540ab-9dd4-4972-842e-b0984743adbf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:57.046902 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d540ab-9dd4-4972-842e-b0984743adbf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:57.047199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d540ab-9dd4-4972-842e-b0984743adbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:57.048062 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d540ab-9dd4-4972-842e-b0984743adbf None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=7bd500f020f143339c5a4fc9f0728ee4) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:57.048374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d540ab-9dd4-4972-842e-b0984743adbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:57.050050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d540ab-9dd4-4972-842e-b0984743adbf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:57.059116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 356/716] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:22:56 2022] GET /identity/v3/role_assignments?user.id=7bd500f020f143339c5a4fc9f0728ee4&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:22:57.075157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b1987-1f90-4afa-b727-ab443bd9854e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:57.221088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b1987-1f90-4afa-b727-ab443bd9854e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:57.263394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4354f-7815-4586-ad20-fead8bae4ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:57.264210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4354f-7815-4586-ad20-fead8bae4ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:57.264492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4354f-7815-4586-ad20-fead8bae4ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:57.264763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4354f-7815-4586-ad20-fead8bae4ac9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:57.265430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4354f-7815-4586-ad20-fead8bae4ac9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:57.266844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4354f-7815-4586-ad20-fead8bae4ac9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:57.273401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-35d4354f-7815-4586-ad20-fead8bae4ac9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:57.287349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:57.288395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:57.289682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 361/717] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:22:57 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:57.295581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d540ab-9dd4-4972-842e-b0984743adbf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:57.418099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d540ab-9dd4-4972-842e-b0984743adbf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:57.427336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:57.461538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9566c-d7a1-4cc9-8c33-7a0e24799b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:57.462237 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9566c-d7a1-4cc9-8c33-7a0e24799b2e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:57.462552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9566c-d7a1-4cc9-8c33-7a0e24799b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:57.462855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9566c-d7a1-4cc9-8c33-7a0e24799b2e None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:57.463926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9566c-d7a1-4cc9-8c33-7a0e24799b2e None admin] RBAC: Adding query filter params (name=network) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:57.464232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9566c-d7a1-4cc9-8c33-7a0e24799b2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:57.465634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9566c-d7a1-4cc9-8c33-7a0e24799b2e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:57.475142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 357/718] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:22:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:57.479884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4354f-7815-4586-ad20-fead8bae4ac9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:57.637178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4354f-7815-4586-ad20-fead8bae4ac9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:57.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b58ce0f-0549-4403-9546-c62370296af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:57.690837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58ce0f-0549-4403-9546-c62370296af2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:57.691122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58ce0f-0549-4403-9546-c62370296af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:57.691393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b58ce0f-0549-4403-9546-c62370296af2 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:57.692297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58ce0f-0549-4403-9546-c62370296af2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:57.692767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58ce0f-0549-4403-9546-c62370296af2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:57.694222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b58ce0f-0549-4403-9546-c62370296af2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:57.702725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 362/719] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:22:57 2022] GET /identity/v3/services?type=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:58.354720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd8eff-05ee-4d31-a707-4b2d53807623 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:58.355254 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd8eff-05ee-4d31-a707-4b2d53807623 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:58.355545 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd8eff-05ee-4d31-a707-4b2d53807623 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:58.357932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 358/720] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:58.369052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26fc5f-c2c3-4893-a5c8-ac5791496b8f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:58.369475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26fc5f-c2c3-4893-a5c8-ac5791496b8f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:58.369888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26fc5f-c2c3-4893-a5c8-ac5791496b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:58.400641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dd26fc5f-c2c3-4893-a5c8-ac5791496b8f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:58.402280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dd26fc5f-c2c3-4893-a5c8-ac5791496b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:58.494080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:58.504945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 363/721] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:58.527743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd8eff-05ee-4d31-a707-4b2d53807623 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:58.663295 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a1efc-cca1-4680-b5eb-e5f62cd1961a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:58.664276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a1efc-cca1-4680-b5eb-e5f62cd1961a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:58.664767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a1efc-cca1-4680-b5eb-e5f62cd1961a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:58.670159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 364/722] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:22:58.671487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd8eff-05ee-4d31-a707-4b2d53807623 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:58.672308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc67ea-9df4-4bf4-a20b-7564270a925f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:58.672809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc67ea-9df4-4bf4-a20b-7564270a925f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:58.673277 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc67ea-9df4-4bf4-a20b-7564270a925f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:58.692588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6bbc67ea-9df4-4bf4-a20b-7564270a925f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:58.694964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6bbc67ea-9df4-4bf4-a20b-7564270a925f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:58.731561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c6ba9-b896-48e8-b45d-e3ba52bd60e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:58.732720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c6ba9-b896-48e8-b45d-e3ba52bd60e8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:58.736161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c6ba9-b896-48e8-b45d-e3ba52bd60e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:58.736738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c6ba9-b896-48e8-b45d-e3ba52bd60e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:58.757389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c6ba9-b896-48e8-b45d-e3ba52bd60e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-188c6ba9-b896-48e8-b45d-e3ba52bd60e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:58.772278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:58.773241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:58.774667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 359/723] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:22:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:58.779610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c6ba9-b896-48e8-b45d-e3ba52bd60e8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:58.808040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 365/724] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 08:22:58.847105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:58.872375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc67ea-9df4-4bf4-a20b-7564270a925f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:58.934560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c6ba9-b896-48e8-b45d-e3ba52bd60e8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:58.975395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a191dec-335f-48c8-abce-0723060c859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:58.976965 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a191dec-335f-48c8-abce-0723060c859a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:58.977677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a191dec-335f-48c8-abce-0723060c859a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:58.978128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a191dec-335f-48c8-abce-0723060c859a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:58.979352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a191dec-335f-48c8-abce-0723060c859a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:58.980274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a191dec-335f-48c8-abce-0723060c859a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:58.982356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a191dec-335f-48c8-abce-0723060c859a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:58.995797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 360/725] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:22:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:58.997432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06a85d-c4f1-4498-bc9c-30ce330471a7 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:58.997918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06a85d-c4f1-4498-bc9c-30ce330471a7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:58.998388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a06a85d-c4f1-4498-bc9c-30ce330471a7 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:58.999965 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 361/726] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:59.001927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06a85d-c4f1-4498-bc9c-30ce330471a7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:59.039941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc67ea-9df4-4bf4-a20b-7564270a925f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:59.070546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:59.084413 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891d52d-d6e3-4b66-8490-fa73db6cbaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:59.085972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d52d-d6e3-4b66-8490-fa73db6cbaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.086399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d52d-d6e3-4b66-8490-fa73db6cbaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.086846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891d52d-d6e3-4b66-8490-fa73db6cbaa8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:59.100113 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891d52d-d6e3-4b66-8490-fa73db6cbaa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:22:59.106743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f891d52d-d6e3-4b66-8490-fa73db6cbaa8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:22:59.110401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:59.111702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:22:59.112693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:59.120220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 366/727] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:22:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:22:59.120220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90177be-c71c-4d3b-8c63-b7d150afd346 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.120405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90177be-c71c-4d3b-8c63-b7d150afd346 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.120580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90177be-c71c-4d3b-8c63-b7d150afd346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:59.124966 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a06a85d-c4f1-4498-bc9c-30ce330471a7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:59.142555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a90177be-c71c-4d3b-8c63-b7d150afd346 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:59.144565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a90177be-c71c-4d3b-8c63-b7d150afd346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:59.162398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73563105-6040-4b52-bde2-8521e55f8c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:59.163422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73563105-6040-4b52-bde2-8521e55f8c5b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.163770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73563105-6040-4b52-bde2-8521e55f8c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.164110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73563105-6040-4b52-bde2-8521e55f8c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:59.176960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73563105-6040-4b52-bde2-8521e55f8c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:59.178854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73563105-6040-4b52-bde2-8521e55f8c5b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-73563105-6040-4b52-bde2-8521e55f8c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:59.198824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:59.199814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:59.201812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:59.201812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:59.201812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:59.201812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:59.204840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 362/728] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:59.205891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73563105-6040-4b52-bde2-8521e55f8c5b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:59.244899 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 367/729] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:59.246205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90177be-c71c-4d3b-8c63-b7d150afd346 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:59.327112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73563105-6040-4b52-bde2-8521e55f8c5b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:59.358834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90177be-c71c-4d3b-8c63-b7d150afd346 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:59.371695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe1a51-2252-4ce6-ba99-82ed701a0087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:59.373143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe1a51-2252-4ce6-ba99-82ed701a0087 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.373635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe1a51-2252-4ce6-ba99-82ed701a0087 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.373887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:59.374270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe1a51-2252-4ce6-ba99-82ed701a0087 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:59.375587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe1a51-2252-4ce6-ba99-82ed701a0087 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:59.376298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe1a51-2252-4ce6-ba99-82ed701a0087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:59.378330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fe1a51-2252-4ce6-ba99-82ed701a0087 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:59.389766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 363/730] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:22:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:59.390861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe1a51-2252-4ce6-ba99-82ed701a0087 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:59.401464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b726d14-8242-4c7b-9d8e-cf32e1de63ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:59.402411 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b726d14-8242-4c7b-9d8e-cf32e1de63ab None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.402928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b726d14-8242-4c7b-9d8e-cf32e1de63ab None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.403378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b726d14-8242-4c7b-9d8e-cf32e1de63ab None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:59.404127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:59.404831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b726d14-8242-4c7b-9d8e-cf32e1de63ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:59.405321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b726d14-8242-4c7b-9d8e-cf32e1de63ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:59.407045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b726d14-8242-4c7b-9d8e-cf32e1de63ab None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:59.424812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 368/731] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:59.425993 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b726d14-8242-4c7b-9d8e-cf32e1de63ab None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:59.525416 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe1a51-2252-4ce6-ba99-82ed701a0087 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:59.541882 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b726d14-8242-4c7b-9d8e-cf32e1de63ab None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:59.562529 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857189f-1cdb-47be-b93b-6fea8160a8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:59.563639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857189f-1cdb-47be-b93b-6fea8160a8da None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.564215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857189f-1cdb-47be-b93b-6fea8160a8da None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.564741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857189f-1cdb-47be-b93b-6fea8160a8da None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:59.565914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857189f-1cdb-47be-b93b-6fea8160a8da None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:59.567723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857189f-1cdb-47be-b93b-6fea8160a8da None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:59.584602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361d6fb-fad4-4af0-a3a3-f20e8e5f166d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:59.585452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361d6fb-fad4-4af0-a3a3-f20e8e5f166d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.585895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361d6fb-fad4-4af0-a3a3-f20e8e5f166d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.586309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361d6fb-fad4-4af0-a3a3-f20e8e5f166d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:59.586439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 364/732] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:22:59 2022] POST /identity/v3/services => generated 251 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:22:59.587216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857189f-1cdb-47be-b93b-6fea8160a8da None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:59.603800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361d6fb-fad4-4af0-a3a3-f20e8e5f166d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:59.605864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361d6fb-fad4-4af0-a3a3-f20e8e5f166d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4361d6fb-fad4-4af0-a3a3-f20e8e5f166d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:22:59.626085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:59.627246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:59.628226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:22:59.628226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:22:59.628226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:22:59.628226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:22:59.630820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 369/733] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:59.632611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4028b-8695-4439-b92f-6d3b80fa6dc3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.633119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4028b-8695-4439-b92f-6d3b80fa6dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.633618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4028b-8695-4439-b92f-6d3b80fa6dc3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:59.635600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 370/734] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:22:59.636627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4028b-8695-4439-b92f-6d3b80fa6dc3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:59.672134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:59.687392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:59.755772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857189f-1cdb-47be-b93b-6fea8160a8da None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:59.763000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4028b-8695-4439-b92f-6d3b80fa6dc3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:59.797577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0088169-1400-4387-934b-24b33e70e62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:59.798665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0088169-1400-4387-934b-24b33e70e62b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.799169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0088169-1400-4387-934b-24b33e70e62b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.799846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0088169-1400-4387-934b-24b33e70e62b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:59.804978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38e894-1b4a-40c8-9cbf-b60ee86fba72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:59.805645 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38e894-1b4a-40c8-9cbf-b60ee86fba72 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.805924 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38e894-1b4a-40c8-9cbf-b60ee86fba72 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.806198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38e894-1b4a-40c8-9cbf-b60ee86fba72 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:59.806960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38e894-1b4a-40c8-9cbf-b60ee86fba72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:22:59.807275 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38e894-1b4a-40c8-9cbf-b60ee86fba72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:59.809207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38e894-1b4a-40c8-9cbf-b60ee86fba72 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:59.810128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0088169-1400-4387-934b-24b33e70e62b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:22:59.811808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0088169-1400-4387-934b-24b33e70e62b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:22:59.821266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 371/735] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:22:59.822846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800be990-9278-430d-bfdd-f27b82314b86 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.823319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800be990-9278-430d-bfdd-f27b82314b86 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.823762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800be990-9278-430d-bfdd-f27b82314b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-c0088169-1400-4387-934b-24b33e70e62b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:22:59.829009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:22:59.829959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:22:59.831043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:22:59.831043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:22:59.831043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:22:59.831043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:22:59.831043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:22:59.831043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:22:59.831043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:22:59.831043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:22:59.831043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:22:59.831043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:22:59.831043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 365/736] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:22:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:22:59.835611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0088169-1400-4387-934b-24b33e70e62b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:59.844284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-800be990-9278-430d-bfdd-f27b82314b86 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:22:59.846007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-800be990-9278-430d-bfdd-f27b82314b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:22:59.930565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 372/737] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:22:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:22:59.931467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800be990-9278-430d-bfdd-f27b82314b86 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:22:59.940882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0088169-1400-4387-934b-24b33e70e62b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:22:59.949237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:59.969815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:22:59.987908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d90d9e-1f0e-421a-a448-eb1e3cbe99c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:22:59.988736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90d9e-1f0e-421a-a448-eb1e3cbe99c1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:22:59.989016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90d9e-1f0e-421a-a448-eb1e3cbe99c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:22:59.989289 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90d9e-1f0e-421a-a448-eb1e3cbe99c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:00.000899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d90d9e-1f0e-421a-a448-eb1e3cbe99c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:00.003577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d90d9e-1f0e-421a-a448-eb1e3cbe99c1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-94d90d9e-1f0e-421a-a448-eb1e3cbe99c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:00.024832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:00.025820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:00.026822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:00.026822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:00.026822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:00.026822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:00.026822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:00.026822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:00.026822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:00.026822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:00.026822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:00.026822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:00.028493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 366/738] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jun 29 08:22:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:00.033781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d90d9e-1f0e-421a-a448-eb1e3cbe99c1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:00.055080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800be990-9278-430d-bfdd-f27b82314b86 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:00.100797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22282c-e5c8-415f-856b-8551d02b3da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:00.102392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22282c-e5c8-415f-856b-8551d02b3da4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:00.102874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22282c-e5c8-415f-856b-8551d02b3da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:00.103339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22282c-e5c8-415f-856b-8551d02b3da4 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:00.104773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22282c-e5c8-415f-856b-8551d02b3da4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:00.105529 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22282c-e5c8-415f-856b-8551d02b3da4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:00.107192 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22282c-e5c8-415f-856b-8551d02b3da4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:00.135174 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 373/739] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jun 29 08:22:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:00.137234 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22282c-e5c8-415f-856b-8551d02b3da4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:00.149713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d90d9e-1f0e-421a-a448-eb1e3cbe99c1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:00.200011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86a859-b9f4-4b42-9d80-54d8327d8ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:00.201631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86a859-b9f4-4b42-9d80-54d8327d8ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:00.202330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86a859-b9f4-4b42-9d80-54d8327d8ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:00.202865 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86a859-b9f4-4b42-9d80-54d8327d8ae4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:00.218210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86a859-b9f4-4b42-9d80-54d8327d8ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:00.222308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3f86a859-b9f4-4b42-9d80-54d8327d8ae4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:00.228407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:00.229374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:00.230198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 367/740] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:23:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:00.230907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86a859-b9f4-4b42-9d80-54d8327d8ae4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:00.254225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22282c-e5c8-415f-856b-8551d02b3da4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:00.277791 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:00.290124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e48448-0c57-4390-959c-2c4aedb8ba77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:00.290644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e48448-0c57-4390-959c-2c4aedb8ba77 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:00.290956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e48448-0c57-4390-959c-2c4aedb8ba77 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:00.291239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e48448-0c57-4390-959c-2c4aedb8ba77 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:00.292073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e48448-0c57-4390-959c-2c4aedb8ba77 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:00.292383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e48448-0c57-4390-959c-2c4aedb8ba77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:00.293854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e48448-0c57-4390-959c-2c4aedb8ba77 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:00.311135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 374/741] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jun 29 08:23:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:00.312374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882d12e-9af0-4e7f-abb8-b04efa48a6c4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:00.312662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882d12e-9af0-4e7f-abb8-b04efa48a6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:00.312941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882d12e-9af0-4e7f-abb8-b04efa48a6c4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:00.318549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 375/742] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:00.318549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882d12e-9af0-4e7f-abb8-b04efa48a6c4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:00.346851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86a859-b9f4-4b42-9d80-54d8327d8ae4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:00.380840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42151cd1-e4f1-40bf-8934-6683dc214d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:00.381522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42151cd1-e4f1-40bf-8934-6683dc214d48 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:00.381787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42151cd1-e4f1-40bf-8934-6683dc214d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:00.382061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42151cd1-e4f1-40bf-8934-6683dc214d48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:00.421614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882d12e-9af0-4e7f-abb8-b04efa48a6c4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:00.458511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42151cd1-e4f1-40bf-8934-6683dc214d48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:00.460695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42151cd1-e4f1-40bf-8934-6683dc214d48 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:00.463208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:00.464701 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95a77f-c1e9-4866-bffc-576482c6196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:00.465622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95a77f-c1e9-4866-bffc-576482c6196a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:00.465899 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95a77f-c1e9-4866-bffc-576482c6196a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:00.466207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95a77f-c1e9-4866-bffc-576482c6196a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:00.467176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95a77f-c1e9-4866-bffc-576482c6196a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:00.467497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95a77f-c1e9-4866-bffc-576482c6196a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:00.469303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b95a77f-c1e9-4866-bffc-576482c6196a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:00.483580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 376/743] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:23:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:00.483580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95a77f-c1e9-4866-bffc-576482c6196a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:00.513043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:00.575903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 368/744] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:00 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:23:00.577191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ee46d-bff0-4197-aeee-4281f67d9592 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:00.577482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ee46d-bff0-4197-aeee-4281f67d9592 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:00.577761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ee46d-bff0-4197-aeee-4281f67d9592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:00.590316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b95a77f-c1e9-4866-bffc-576482c6196a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:00.594675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9f1ee46d-bff0-4197-aeee-4281f67d9592 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:00.595941 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9f1ee46d-bff0-4197-aeee-4281f67d9592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:00.625069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c23791-0414-4ff1-9f9e-3d0fbd16c361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:00.625545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c23791-0414-4ff1-9f9e-3d0fbd16c361 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:00.625824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c23791-0414-4ff1-9f9e-3d0fbd16c361 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:00.626102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c23791-0414-4ff1-9f9e-3d0fbd16c361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:00.665001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 369/745] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 08:23:00.666250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ee46d-bff0-4197-aeee-4281f67d9592 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:00.703046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:00.713195 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c23791-0414-4ff1-9f9e-3d0fbd16c361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:00.715436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c23791-0414-4ff1-9f9e-3d0fbd16c361 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:00.742719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:00.772709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ee46d-bff0-4197-aeee-4281f67d9592 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:00.809442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ab308-5f5f-47e9-9f1c-c0e71a560460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:00.810292 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ab308-5f5f-47e9-9f1c-c0e71a560460 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:00.810581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ab308-5f5f-47e9-9f1c-c0e71a560460 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:00.810899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ab308-5f5f-47e9-9f1c-c0e71a560460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:00.823585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ab308-5f5f-47e9-9f1c-c0e71a560460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:00.825137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ab308-5f5f-47e9-9f1c-c0e71a560460 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-7d1ab308-5f5f-47e9-9f1c-c0e71a560460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:00.846589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:00.847775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:00.848652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:00.848652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:00.848652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:00.848652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:00.851094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 370/746] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:00.852336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ab308-5f5f-47e9-9f1c-c0e71a560460 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:00.953310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 377/747] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:00 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 08:23:00.954149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c23791-0414-4ff1-9f9e-3d0fbd16c361 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:00.968263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ab308-5f5f-47e9-9f1c-c0e71a560460 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:00.995105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:01.005270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59390a-6eee-4a5c-a600-950edb6182f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:01.006309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59390a-6eee-4a5c-a600-950edb6182f5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:01.006638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59390a-6eee-4a5c-a600-950edb6182f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:01.006893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59390a-6eee-4a5c-a600-950edb6182f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:01.017935 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59390a-6eee-4a5c-a600-950edb6182f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:01.019751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59390a-6eee-4a5c-a600-950edb6182f5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6b59390a-6eee-4a5c-a600-950edb6182f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:01.037227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:01.038107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:01.039037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:01.039037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:01.039037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:01.039037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:01.039037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 371/748] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:01.039315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59390a-6eee-4a5c-a600-950edb6182f5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:01.058257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:01.065661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c23791-0414-4ff1-9f9e-3d0fbd16c361 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:01.101639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699b344-9924-4b21-8d7d-0c123414dbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:01.102351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699b344-9924-4b21-8d7d-0c123414dbb6 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:01.102636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699b344-9924-4b21-8d7d-0c123414dbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:01.102943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699b344-9924-4b21-8d7d-0c123414dbb6 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:01.103671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699b344-9924-4b21-8d7d-0c123414dbb6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:01.105052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699b344-9924-4b21-8d7d-0c123414dbb6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:01.121769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 378/749] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:23:00 2022] POST /identity/v3/services => generated 240 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:01.122568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699b344-9924-4b21-8d7d-0c123414dbb6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:01.157405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59390a-6eee-4a5c-a600-950edb6182f5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:01.208393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fcb4c-9deb-4e43-995a-10eb57af8b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:01.209471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fcb4c-9deb-4e43-995a-10eb57af8b1e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:01.209845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fcb4c-9deb-4e43-995a-10eb57af8b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:01.210288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fcb4c-9deb-4e43-995a-10eb57af8b1e None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:01.211270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fcb4c-9deb-4e43-995a-10eb57af8b1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:01.211752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fcb4c-9deb-4e43-995a-10eb57af8b1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:01.213890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fcb4c-9deb-4e43-995a-10eb57af8b1e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:01.233270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 372/750] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:01.233270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fcb4c-9deb-4e43-995a-10eb57af8b1e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:01.272891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:01.273474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699b344-9924-4b21-8d7d-0c123414dbb6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:01.325661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c922e80-9adc-4777-9a3f-019bf0e0875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:01.326602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c922e80-9adc-4777-9a3f-019bf0e0875f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:01.327001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c922e80-9adc-4777-9a3f-019bf0e0875f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:01.327359 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c922e80-9adc-4777-9a3f-019bf0e0875f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:01.338952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c922e80-9adc-4777-9a3f-019bf0e0875f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:01.341027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c922e80-9adc-4777-9a3f-019bf0e0875f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:01.356383 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fcb4c-9deb-4e43-995a-10eb57af8b1e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-8c922e80-9adc-4777-9a3f-019bf0e0875f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:01.360566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:01.361501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:01.362595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:01.362595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:01.362595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:01.362595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:01.365178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 379/751] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:01.365178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c922e80-9adc-4777-9a3f-019bf0e0875f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:01.375031 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:01.407381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9a84b-21cf-4323-9a8f-bf627ce5fcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:01.407967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9a84b-21cf-4323-9a8f-bf627ce5fcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:01.407967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9a84b-21cf-4323-9a8f-bf627ce5fcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:01.408106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9a84b-21cf-4323-9a8f-bf627ce5fcb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:01.408867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9a84b-21cf-4323-9a8f-bf627ce5fcb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:01.409179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9a84b-21cf-4323-9a8f-bf627ce5fcb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:01.410564 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9a84b-21cf-4323-9a8f-bf627ce5fcb9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:01.427867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 373/752] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:01.429127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9a84b-21cf-4323-9a8f-bf627ce5fcb9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:01.488981 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c922e80-9adc-4777-9a3f-019bf0e0875f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:01.525365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910fcb6-4ed9-4a1e-8752-d23ecef8c7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:01.526216 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910fcb6-4ed9-4a1e-8752-d23ecef8c7ed None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:01.526552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910fcb6-4ed9-4a1e-8752-d23ecef8c7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:01.526825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910fcb6-4ed9-4a1e-8752-d23ecef8c7ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:01.536615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910fcb6-4ed9-4a1e-8752-d23ecef8c7ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:01.537761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910fcb6-4ed9-4a1e-8752-d23ecef8c7ed None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0910fcb6-4ed9-4a1e-8752-d23ecef8c7ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:01.553123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:01.554093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:01.555469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:01.555469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:01.555469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:01.555469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:01.555469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:01.555469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:01.555469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:01.555469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:01.555469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:23:01.555469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:01.559065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 380/753] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jun 29 08:23:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:01.559710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910fcb6-4ed9-4a1e-8752-d23ecef8c7ed None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:01.562077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:01.571629 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9a84b-21cf-4323-9a8f-bf627ce5fcb9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:01.620682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbc124-7e7d-41bb-8c92-1b91d717212b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:01.622208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc124-7e7d-41bb-8c92-1b91d717212b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:01.622787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc124-7e7d-41bb-8c92-1b91d717212b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:01.623238 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc124-7e7d-41bb-8c92-1b91d717212b None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:01.624805 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbc124-7e7d-41bb-8c92-1b91d717212b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:01.625296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbc124-7e7d-41bb-8c92-1b91d717212b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:01.627544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbc124-7e7d-41bb-8c92-1b91d717212b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:01.631739 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:01.650220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 374/754] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:01.651661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbc124-7e7d-41bb-8c92-1b91d717212b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:01.673485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910fcb6-4ed9-4a1e-8752-d23ecef8c7ed None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:01.710078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462ee3-dd43-4000-86f6-5d96684b3814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:01.711054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462ee3-dd43-4000-86f6-5d96684b3814 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:01.711449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462ee3-dd43-4000-86f6-5d96684b3814 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:01.711808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462ee3-dd43-4000-86f6-5d96684b3814 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:01.719531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1462ee3-dd43-4000-86f6-5d96684b3814 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:01.721233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1462ee3-dd43-4000-86f6-5d96684b3814 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e1462ee3-dd43-4000-86f6-5d96684b3814 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:01.736491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:01.737590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:01.738613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:01.738613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:01.738613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:01.738613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:01.738613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:01.738613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:01.738613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:01.738613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:01.739903 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 381/755] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:01.745409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462ee3-dd43-4000-86f6-5d96684b3814 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:01.782341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbc124-7e7d-41bb-8c92-1b91d717212b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:01.820029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626da409-c2bf-4358-a202-447fdf5eb546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:01.820980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626da409-c2bf-4358-a202-447fdf5eb546 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:01.821260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626da409-c2bf-4358-a202-447fdf5eb546 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:01.821537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626da409-c2bf-4358-a202-447fdf5eb546 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:01.822436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626da409-c2bf-4358-a202-447fdf5eb546 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:01.822779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626da409-c2bf-4358-a202-447fdf5eb546 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:01.824269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626da409-c2bf-4358-a202-447fdf5eb546 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:01.854846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462ee3-dd43-4000-86f6-5d96684b3814 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:01.886281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:01.897650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad80ab0-c64e-471f-96b2-661dd3c7a51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:01.898475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad80ab0-c64e-471f-96b2-661dd3c7a51e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:01.898779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad80ab0-c64e-471f-96b2-661dd3c7a51e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:01.899054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad80ab0-c64e-471f-96b2-661dd3c7a51e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:01.907275 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad80ab0-c64e-471f-96b2-661dd3c7a51e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:01.908867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad80ab0-c64e-471f-96b2-661dd3c7a51e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7ad80ab0-c64e-471f-96b2-661dd3c7a51e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:01.925015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:01.925965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:01.932097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:01.932097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:01.932097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:01.932097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:01.932097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:01.932097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:01.932097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:01.932097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:01.932097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 382/756] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:01.932097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad80ab0-c64e-471f-96b2-661dd3c7a51e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:01.962425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 375/757] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jun 29 08:23:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:01.964699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97ea7-f1f4-48c8-aa64-e634fdc54b1b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:01.965053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97ea7-f1f4-48c8-aa64-e634fdc54b1b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:01.965396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97ea7-f1f4-48c8-aa64-e634fdc54b1b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:01.970989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 376/758] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:01.970989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f97ea7-f1f4-48c8-aa64-e634fdc54b1b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:01.983359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:02.046745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad80ab0-c64e-471f-96b2-661dd3c7a51e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:02.078453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f97ea7-f1f4-48c8-aa64-e634fdc54b1b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:02.082878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46972-fedf-41e9-84c6-88d8ade5fec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:02.083555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46972-fedf-41e9-84c6-88d8ade5fec3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:02.083830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46972-fedf-41e9-84c6-88d8ade5fec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:02.084104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46972-fedf-41e9-84c6-88d8ade5fec3 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:02.084904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a46972-fedf-41e9-84c6-88d8ade5fec3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:02.085218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a46972-fedf-41e9-84c6-88d8ade5fec3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:02.086660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a46972-fedf-41e9-84c6-88d8ade5fec3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:02.099368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 383/759] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:02.100014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46972-fedf-41e9-84c6-88d8ade5fec3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:02.121064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480f214-dca1-450b-84a5-63119ccaa711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:02.121996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480f214-dca1-450b-84a5-63119ccaa711 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:02.122421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480f214-dca1-450b-84a5-63119ccaa711 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:02.122882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480f214-dca1-450b-84a5-63119ccaa711 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:02.123940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480f214-dca1-450b-84a5-63119ccaa711 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:02.124401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480f214-dca1-450b-84a5-63119ccaa711 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:02.126101 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480f214-dca1-450b-84a5-63119ccaa711 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:02.139593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 377/760] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:02.141170 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cb81a-ac41-4c21-b504-324dc453c2ac None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:02.141658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cb81a-ac41-4c21-b504-324dc453c2ac None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:02.141936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cb81a-ac41-4c21-b504-324dc453c2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:02.152942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:02.165255 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-515cb81a-ac41-4c21-b504-324dc453c2ac None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:02.166456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-515cb81a-ac41-4c21-b504-324dc453c2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:02.215682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46972-fedf-41e9-84c6-88d8ade5fec3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:02.237970 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 378/761] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:02.238855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cb81a-ac41-4c21-b504-324dc453c2ac None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:02.241093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:02.255022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ae4f8-f705-427d-b233-4d4dd1432f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:02.256086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ae4f8-f705-427d-b233-4d4dd1432f7d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:02.256726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ae4f8-f705-427d-b233-4d4dd1432f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:02.257166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ae4f8-f705-427d-b233-4d4dd1432f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:02.270134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ae4f8-f705-427d-b233-4d4dd1432f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:02.272086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ae4f8-f705-427d-b233-4d4dd1432f7d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5a6ae4f8-f705-427d-b233-4d4dd1432f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:02.290913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:02.291860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:02.292728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:02.292728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:02.292728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:02.292728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:02.294774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 384/762] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:02.295883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ae4f8-f705-427d-b233-4d4dd1432f7d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:02.358131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cb81a-ac41-4c21-b504-324dc453c2ac None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:02.393065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:02.393540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bd847-0453-43c6-adf9-63f3dc4ea5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:02.394208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd847-0453-43c6-adf9-63f3dc4ea5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:02.394525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd847-0453-43c6-adf9-63f3dc4ea5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:02.394836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd847-0453-43c6-adf9-63f3dc4ea5b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:02.395794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bd847-0453-43c6-adf9-63f3dc4ea5b7 None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=3b38411347754c058eb6640829078e1d) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:02.396180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bd847-0453-43c6-adf9-63f3dc4ea5b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:02.397631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8bd847-0453-43c6-adf9-63f3dc4ea5b7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:02.399316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ae4f8-f705-427d-b233-4d4dd1432f7d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:02.403291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 379/763] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:02 2022] GET /identity/v3/role_assignments?user.id=3b38411347754c058eb6640829078e1d&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:02.404071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bd847-0453-43c6-adf9-63f3dc4ea5b7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:02.447131 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ba63e-80f5-46c2-a17a-b49ecd5f5aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:02.447865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ba63e-80f5-46c2-a17a-b49ecd5f5aab None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:02.448155 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ba63e-80f5-46c2-a17a-b49ecd5f5aab None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:02.448431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ba63e-80f5-46c2-a17a-b49ecd5f5aab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:02.449327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ba63e-80f5-46c2-a17a-b49ecd5f5aab None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=f320b56bf7ff4c36bbd9730e688612d5) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:02.449649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ba63e-80f5-46c2-a17a-b49ecd5f5aab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:02.459785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ba63e-80f5-46c2-a17a-b49ecd5f5aab None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:02.473242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 385/764] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:02 2022] GET /identity/v3/role_assignments?user.id=f320b56bf7ff4c36bbd9730e688612d5&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:02.473242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ba63e-80f5-46c2-a17a-b49ecd5f5aab None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:02.529439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:02.609024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bd847-0453-43c6-adf9-63f3dc4ea5b7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:02.660481 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ba63e-80f5-46c2-a17a-b49ecd5f5aab None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:02.671443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203eba1a-6821-4240-b9ae-4f3423e878e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:02.672279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eba1a-6821-4240-b9ae-4f3423e878e2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:02.672559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eba1a-6821-4240-b9ae-4f3423e878e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:02.672830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eba1a-6821-4240-b9ae-4f3423e878e2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:02.673496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203eba1a-6821-4240-b9ae-4f3423e878e2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:02.675446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203eba1a-6821-4240-b9ae-4f3423e878e2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-203eba1a-6821-4240-b9ae-4f3423e878e2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:02.697530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:02.698500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 08:23:02.699997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:02.700933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac126c0-6d10-40de-9328-cd337892b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:02.700933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac126c0-6d10-40de-9328-cd337892b489 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:02.700933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac126c0-6d10-40de-9328-cd337892b489 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:02.701174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 380/765] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Jun 29 08:23:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:02.701256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac126c0-6d10-40de-9328-cd337892b489 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:02.701256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac126c0-6d10-40de-9328-cd337892b489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:02.701256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac126c0-6d10-40de-9328-cd337892b489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:02.710728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac126c0-6d10-40de-9328-cd337892b489 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:02.712669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203eba1a-6821-4240-b9ae-4f3423e878e2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:02.715750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 386/766] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:02.724576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac126c0-6d10-40de-9328-cd337892b489 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:02.771831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:02.855075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203eba1a-6821-4240-b9ae-4f3423e878e2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:02.893658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:02.908141 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc3caa-dfe1-43c2-9251-8a5859b42a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:02.909522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc3caa-dfe1-43c2-9251-8a5859b42a9c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:02.909807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc3caa-dfe1-43c2-9251-8a5859b42a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:02.910083 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc3caa-dfe1-43c2-9251-8a5859b42a9c None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:02.910897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc3caa-dfe1-43c2-9251-8a5859b42a9c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:02.911152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc3caa-dfe1-43c2-9251-8a5859b42a9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:02.912540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc3caa-dfe1-43c2-9251-8a5859b42a9c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:02.922807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 381/767] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jun 29 08:23:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:02.928145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac126c0-6d10-40de-9328-cd337892b489 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:02.930393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc3caa-dfe1-43c2-9251-8a5859b42a9c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:02.983137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d7777-68b9-4e62-ad33-f80620519bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:02.984323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7777-68b9-4e62-ad33-f80620519bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:02.984605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7777-68b9-4e62-ad33-f80620519bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:02.984879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7777-68b9-4e62-ad33-f80620519bd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:02.992246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d7777-68b9-4e62-ad33-f80620519bd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:02.994073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d7777-68b9-4e62-ad33-f80620519bd9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5c5d7777-68b9-4e62-ad33-f80620519bd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:03.014562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:03.015536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:03.016419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:03.016419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:03.016419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:03.016419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:03.016419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:03.016419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:03.016419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:03.016419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:03.024553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 387/768] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:03.026193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d7777-68b9-4e62-ad33-f80620519bd9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:03.109921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc3caa-dfe1-43c2-9251-8a5859b42a9c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:03.188811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d36a2c-fbb1-41af-8bcb-4b953ca59cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:03.190225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d36a2c-fbb1-41af-8bcb-4b953ca59cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:03.191802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d36a2c-fbb1-41af-8bcb-4b953ca59cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:03.192365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d36a2c-fbb1-41af-8bcb-4b953ca59cd0 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:03.192842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:03.195696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d36a2c-fbb1-41af-8bcb-4b953ca59cd0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:03.197811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d36a2c-fbb1-41af-8bcb-4b953ca59cd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:03.200539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d36a2c-fbb1-41af-8bcb-4b953ca59cd0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:03.213165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 382/769] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jun 29 08:23:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:03.218109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d7777-68b9-4e62-ad33-f80620519bd9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:03.218868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d36a2c-fbb1-41af-8bcb-4b953ca59cd0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:03.256463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852433b-02f8-408c-8731-1880e38f17b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:03.257530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852433b-02f8-408c-8731-1880e38f17b8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:03.258025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852433b-02f8-408c-8731-1880e38f17b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:03.258552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852433b-02f8-408c-8731-1880e38f17b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:03.259679 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852433b-02f8-408c-8731-1880e38f17b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:03.260188 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852433b-02f8-408c-8731-1880e38f17b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:03.262097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852433b-02f8-408c-8731-1880e38f17b8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:03.276080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 388/770] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:03.284057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852433b-02f8-408c-8731-1880e38f17b8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:03.289634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:03.331766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d36a2c-fbb1-41af-8bcb-4b953ca59cd0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:03.366037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c0a66-27d9-463a-8403-67b5d46876d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:03.366717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c0a66-27d9-463a-8403-67b5d46876d9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:03.367019 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c0a66-27d9-463a-8403-67b5d46876d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:03.367291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c0a66-27d9-463a-8403-67b5d46876d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:03.368089 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c0a66-27d9-463a-8403-67b5d46876d9 None admin] RBAC: Adding query filter params (interface=public, service_id=c67fb4eb02be47ebb3b2c7e8ddfe81e7, region_id=RegionOne) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:03.368399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c0a66-27d9-463a-8403-67b5d46876d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:03.369766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c0a66-27d9-463a-8403-67b5d46876d9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:03.378434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 383/771] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jun 29 08:23:03 2022] GET /identity/v3/endpoints?service_id=c67fb4eb02be47ebb3b2c7e8ddfe81e7&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:03.379918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1975778-9e7d-4ee6-8c67-38e12bad44a2 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:03.380204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1975778-9e7d-4ee6-8c67-38e12bad44a2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:03.380481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1975778-9e7d-4ee6-8c67-38e12bad44a2 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:03.381949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 384/772] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:03.383299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1975778-9e7d-4ee6-8c67-38e12bad44a2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:03.399738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852433b-02f8-408c-8731-1880e38f17b8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:03.433507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa646bca-b884-4daf-bcbc-0e3a672b96ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:03.434763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa646bca-b884-4daf-bcbc-0e3a672b96ef None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:03.435244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa646bca-b884-4daf-bcbc-0e3a672b96ef None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:03.435709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa646bca-b884-4daf-bcbc-0e3a672b96ef None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/7bd500f020f143339c5a4fc9f0728ee4/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:03.437894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa646bca-b884-4daf-bcbc-0e3a672b96ef None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=7bd500f020f143339c5a4fc9f0728ee4, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:03.439685 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa646bca-b884-4daf-bcbc-0e3a672b96ef None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:03.468925 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 389/773] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:23:03 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/7bd500f020f143339c5a4fc9f0728ee4/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:23:03.476427 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd976e47-fe14-4ac4-a946-0d5c500e441d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:03.476745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd976e47-fe14-4ac4-a946-0d5c500e441d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:03.476973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd976e47-fe14-4ac4-a946-0d5c500e441d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:03.505854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:03.518704 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fd976e47-fe14-4ac4-a946-0d5c500e441d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:03.519913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fd976e47-fe14-4ac4-a946-0d5c500e441d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:03.521392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1975778-9e7d-4ee6-8c67-38e12bad44a2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:03.557241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:03.578875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10904dc-1d11-47bb-a096-c27affa206d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:03.579559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10904dc-1d11-47bb-a096-c27affa206d2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:03.579837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10904dc-1d11-47bb-a096-c27affa206d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:03.580161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10904dc-1d11-47bb-a096-c27affa206d2 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:03.580864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10904dc-1d11-47bb-a096-c27affa206d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:03.582289 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10904dc-1d11-47bb-a096-c27affa206d2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:03.594269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 385/774] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jun 29 08:23:03 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:03.633156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 390/775] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:03.657672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10904dc-1d11-47bb-a096-c27affa206d2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:03.774814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10904dc-1d11-47bb-a096-c27affa206d2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:03.825485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc0b3a-326f-4fcb-a6d8-f0562f40495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:03.826714 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc0b3a-326f-4fcb-a6d8-f0562f40495d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:03.827197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc0b3a-326f-4fcb-a6d8-f0562f40495d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:03.827666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc0b3a-326f-4fcb-a6d8-f0562f40495d None admin] PATH_INFO: `/v3/services/network` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:03.828520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc0b3a-326f-4fcb-a6d8-f0562f40495d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:03.830173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc0b3a-326f-4fcb-a6d8-f0562f40495d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-44cc0b3a-326f-4fcb-a6d8-f0562f40495d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:03.843635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:03.844649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 08:23:03.845491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:03.848405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 386/776] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:03 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:03.854061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd976e47-fe14-4ac4-a946-0d5c500e441d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:03.965022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd976e47-fe14-4ac4-a946-0d5c500e441d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:03.995713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:04.005797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd05d0-30e0-4d54-b569-72a0af65c483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:04.006781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd05d0-30e0-4d54-b569-72a0af65c483 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:04.007263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd05d0-30e0-4d54-b569-72a0af65c483 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:04.007771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd05d0-30e0-4d54-b569-72a0af65c483 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:04.008733 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd05d0-30e0-4d54-b569-72a0af65c483 None admin] RBAC: Adding query filter params (name=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:04.009237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd05d0-30e0-4d54-b569-72a0af65c483 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:04.010842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccd05d0-30e0-4d54-b569-72a0af65c483 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:04.016337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 391/777] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:23:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:04.023786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc0b3a-326f-4fcb-a6d8-f0562f40495d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:04.091585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:04.146856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc0b3a-326f-4fcb-a6d8-f0562f40495d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:04.181894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb67255-808d-4fc1-9ec9-1b2e45458273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:04.182618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb67255-808d-4fc1-9ec9-1b2e45458273 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:04.182927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb67255-808d-4fc1-9ec9-1b2e45458273 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:04.183204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb67255-808d-4fc1-9ec9-1b2e45458273 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:04.183956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb67255-808d-4fc1-9ec9-1b2e45458273 None admin] RBAC: Adding query filter params (type=network) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:04.184270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb67255-808d-4fc1-9ec9-1b2e45458273 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:04.185648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb67255-808d-4fc1-9ec9-1b2e45458273 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:04.192662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 387/778] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:23:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:04.198342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd05d0-30e0-4d54-b569-72a0af65c483 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:04.305270 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccd05d0-30e0-4d54-b569-72a0af65c483 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:04.339773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762d677-cc20-4cc0-a0ae-2cf504e44103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:04.340443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762d677-cc20-4cc0-a0ae-2cf504e44103 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:04.340717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762d677-cc20-4cc0-a0ae-2cf504e44103 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:04.341049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762d677-cc20-4cc0-a0ae-2cf504e44103 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:04.341906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762d677-cc20-4cc0-a0ae-2cf504e44103 None admin] RBAC: Adding query filter params (interface=public, service_id=6b54286a6bbf4966b2637e53eb3518f7, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:04.342202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762d677-cc20-4cc0-a0ae-2cf504e44103 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:04.343606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762d677-cc20-4cc0-a0ae-2cf504e44103 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:04.353040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 392/779] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jun 29 08:23:04 2022] GET /identity/v3/endpoints?service_id=6b54286a6bbf4966b2637e53eb3518f7&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:04.358478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb67255-808d-4fc1-9ec9-1b2e45458273 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:04.466717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb67255-808d-4fc1-9ec9-1b2e45458273 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:04.478051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:04.500182 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b7080-1d3c-4a56-99d0-fc72d604f7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:04.500853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b7080-1d3c-4a56-99d0-fc72d604f7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:04.501131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b7080-1d3c-4a56-99d0-fc72d604f7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:04.501405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b7080-1d3c-4a56-99d0-fc72d604f7b4 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:04.502127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b7080-1d3c-4a56-99d0-fc72d604f7b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:04.503537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b7080-1d3c-4a56-99d0-fc72d604f7b4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:04.509377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 388/780] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jun 29 08:23:04 2022] GET /identity/v3/services => generated 772 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:04.836347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ede3-5b5d-4fa5-9488-aefdefdc08ba None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:04.836704 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ede3-5b5d-4fa5-9488-aefdefdc08ba None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:04.836909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ede3-5b5d-4fa5-9488-aefdefdc08ba None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:04.838718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 393/781] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:04.844216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f01ce-2685-48a2-8ea6-1c0166e05789 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:04.844673 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f01ce-2685-48a2-8ea6-1c0166e05789 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:04.844760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f01ce-2685-48a2-8ea6-1c0166e05789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:04.867756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3f4f01ce-2685-48a2-8ea6-1c0166e05789 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:04.869016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3f4f01ce-2685-48a2-8ea6-1c0166e05789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:04.943035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 389/782] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:04.977293 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883ede3-5b5d-4fa5-9488-aefdefdc08ba None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:05.001854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eaa8ba-9b5e-4291-a595-b2f8d2f5a53f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:05.002809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eaa8ba-9b5e-4291-a595-b2f8d2f5a53f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:05.003121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eaa8ba-9b5e-4291-a595-b2f8d2f5a53f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:05.004506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 390/783] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:05.018018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffa112-6178-495a-bb68-51362e30f07f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:05.018325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffa112-6178-495a-bb68-51362e30f07f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:05.018579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffa112-6178-495a-bb68-51362e30f07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:05.047718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d3ffa112-6178-495a-bb68-51362e30f07f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:05.048991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d3ffa112-6178-495a-bb68-51362e30f07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:05.059091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:05.096922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883ede3-5b5d-4fa5-9488-aefdefdc08ba None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:05.134035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3dacd-6011-4bb6-9cfa-c37cda525ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:05.134895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3dacd-6011-4bb6-9cfa-c37cda525ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:05.135177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3dacd-6011-4bb6-9cfa-c37cda525ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:05.135596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3dacd-6011-4bb6-9cfa-c37cda525ea0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:05.146802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3dacd-6011-4bb6-9cfa-c37cda525ea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5cb3dacd-6011-4bb6-9cfa-c37cda525ea0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:05.154948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:05.155867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:05.156682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 394/784] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:23:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:05.162527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3dacd-6011-4bb6-9cfa-c37cda525ea0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:05.205039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 391/785] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:05.242434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffa112-6178-495a-bb68-51362e30f07f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:05.321960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:05.342830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3dacd-6011-4bb6-9cfa-c37cda525ea0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:05.387143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:05.416058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f4651-0d20-4da9-a11e-8483d67ce6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:05.416738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f4651-0d20-4da9-a11e-8483d67ce6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:05.417014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f4651-0d20-4da9-a11e-8483d67ce6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:05.417286 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f4651-0d20-4da9-a11e-8483d67ce6f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:05.418039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f4651-0d20-4da9-a11e-8483d67ce6f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:05.418345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f4651-0d20-4da9-a11e-8483d67ce6f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:05.422911 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f4651-0d20-4da9-a11e-8483d67ce6f2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:05.438608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffa112-6178-495a-bb68-51362e30f07f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:05.461074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 395/786] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:23:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:05.474534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d76d1-0d6d-455f-9f22-ef10cbdf230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:05.475387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d76d1-0d6d-455f-9f22-ef10cbdf230a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:05.475669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d76d1-0d6d-455f-9f22-ef10cbdf230a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:05.475773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f4651-0d20-4da9-a11e-8483d67ce6f2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:05.475990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d76d1-0d6d-455f-9f22-ef10cbdf230a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:05.486911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d76d1-0d6d-455f-9f22-ef10cbdf230a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-712d76d1-0d6d-455f-9f22-ef10cbdf230a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:05.501473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:05.502495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:05.503484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 392/787] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:23:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:05.510343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d76d1-0d6d-455f-9f22-ef10cbdf230a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:05.676564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f4651-0d20-4da9-a11e-8483d67ce6f2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:05.714241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce783412-638c-4b94-924e-f9c5dbf587b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:05.715082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce783412-638c-4b94-924e-f9c5dbf587b1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:05.715373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce783412-638c-4b94-924e-f9c5dbf587b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:05.715653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce783412-638c-4b94-924e-f9c5dbf587b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:05.728508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce783412-638c-4b94-924e-f9c5dbf587b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:05.730004 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce783412-638c-4b94-924e-f9c5dbf587b1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ce783412-638c-4b94-924e-f9c5dbf587b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:05.748382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:05.749310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:05.750197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:05.750197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:05.750197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:05.750197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:05.750197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 396/788] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:05.755058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:05.758105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce783412-638c-4b94-924e-f9c5dbf587b1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:05.768647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d76d1-0d6d-455f-9f22-ef10cbdf230a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:05.797510 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:05.871551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce783412-638c-4b94-924e-f9c5dbf587b1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:05.887534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a858d-1d9f-4968-b7d1-0e4be8eb1f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:05.888241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a858d-1d9f-4968-b7d1-0e4be8eb1f5f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:05.888513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a858d-1d9f-4968-b7d1-0e4be8eb1f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:05.888785 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291a858d-1d9f-4968-b7d1-0e4be8eb1f5f None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:05.889563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a858d-1d9f-4968-b7d1-0e4be8eb1f5f None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:05.889877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a858d-1d9f-4968-b7d1-0e4be8eb1f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:05.891353 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291a858d-1d9f-4968-b7d1-0e4be8eb1f5f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:05.910116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2e2bf-a1f1-4468-9343-d05f3082d5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:05.910823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2e2bf-a1f1-4468-9343-d05f3082d5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:05.911111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2e2bf-a1f1-4468-9343-d05f3082d5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:05.911392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2e2bf-a1f1-4468-9343-d05f3082d5e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:05.911753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 393/789] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:23:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:05.912128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2e2bf-a1f1-4468-9343-d05f3082d5e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:05.912439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2e2bf-a1f1-4468-9343-d05f3082d5e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:05.913827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2e2bf-a1f1-4468-9343-d05f3082d5e6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:05.917806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a858d-1d9f-4968-b7d1-0e4be8eb1f5f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:05.927064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 397/790] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:05.944253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2e2bf-a1f1-4468-9343-d05f3082d5e6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:06.067268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:06.078636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2e2bf-a1f1-4468-9343-d05f3082d5e6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:06.083590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291a858d-1d9f-4968-b7d1-0e4be8eb1f5f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:06.119696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d372020-ab7c-4cfa-bea8-56597cb51412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:06.120861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d372020-ab7c-4cfa-bea8-56597cb51412 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.121337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d372020-ab7c-4cfa-bea8-56597cb51412 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.121804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d372020-ab7c-4cfa-bea8-56597cb51412 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.130044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a081bc4a-ce9c-49b7-a216-52adf09a2660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:06.130550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d372020-ab7c-4cfa-bea8-56597cb51412 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:06.130909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081bc4a-ce9c-49b7-a216-52adf09a2660 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.131194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081bc4a-ce9c-49b7-a216-52adf09a2660 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.131475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081bc4a-ce9c-49b7-a216-52adf09a2660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.132882 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d372020-ab7c-4cfa-bea8-56597cb51412 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:06.144004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081bc4a-ce9c-49b7-a216-52adf09a2660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:06.145478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a081bc4a-ce9c-49b7-a216-52adf09a2660 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7d372020-ab7c-4cfa-bea8-56597cb51412 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:06.148304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:06.149231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:06.150159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:06.150159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:06.150159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:06.150159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:06.150159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:06.150159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:06.150159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:06.150159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:06.150159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:23:06.150159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:06.152873 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 398/791] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:23:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:06.154355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ec115-b51d-4d37-9f2e-d2efc1e6b164 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.154867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ec115-b51d-4d37-9f2e-d2efc1e6b164 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.155340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ec115-b51d-4d37-9f2e-d2efc1e6b164 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.160990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 399/792] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:06.162009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ec115-b51d-4d37-9f2e-d2efc1e6b164 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a081bc4a-ce9c-49b7-a216-52adf09a2660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:06.164373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:06.165225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:06.166099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:06.166099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:06.166099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:06.166099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:06.166099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 394/793] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:06.171203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967b74-5013-49f6-8d44-a050e3ca47b0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.171515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967b74-5013-49f6-8d44-a050e3ca47b0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.171786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa967b74-5013-49f6-8d44-a050e3ca47b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.191382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fa967b74-5013-49f6-8d44-a050e3ca47b0 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:06.192481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fa967b74-5013-49f6-8d44-a050e3ca47b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:06.200485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:06.275006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 395/794] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:06.275006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa967b74-5013-49f6-8d44-a050e3ca47b0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:06.283852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ec115-b51d-4d37-9f2e-d2efc1e6b164 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:06.323711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fcf16b-39ce-4b27-8e31-b97d996061e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:06.324388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcf16b-39ce-4b27-8e31-b97d996061e1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.324674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcf16b-39ce-4b27-8e31-b97d996061e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.324956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcf16b-39ce-4b27-8e31-b97d996061e1 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.325815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fcf16b-39ce-4b27-8e31-b97d996061e1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:06.326132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fcf16b-39ce-4b27-8e31-b97d996061e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:06.327601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fcf16b-39ce-4b27-8e31-b97d996061e1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:06.346733 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 400/795] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jun 29 08:23:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:06.348047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce0e19-21e8-4746-bccb-2b39fdd89e06 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.348338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce0e19-21e8-4746-bccb-2b39fdd89e06 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.348615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce0e19-21e8-4746-bccb-2b39fdd89e06 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.349997 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 401/796] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:06.351447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce0e19-21e8-4746-bccb-2b39fdd89e06 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:06.370786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:06.387321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa967b74-5013-49f6-8d44-a050e3ca47b0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:06.424918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298fbcb-269b-476c-8bea-e543c366164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:06.425696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298fbcb-269b-476c-8bea-e543c366164b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.426040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298fbcb-269b-476c-8bea-e543c366164b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.426337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298fbcb-269b-476c-8bea-e543c366164b None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.427085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298fbcb-269b-476c-8bea-e543c366164b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:06.427723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298fbcb-269b-476c-8bea-e543c366164b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:06.428816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298fbcb-269b-476c-8bea-e543c366164b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:06.440953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 396/797] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:06.441660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298fbcb-269b-476c-8bea-e543c366164b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:06.444263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:06.476324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce0e19-21e8-4746-bccb-2b39fdd89e06 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:06.512203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544025a9-c3ef-4b22-b2d9-65be5a176dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:06.513011 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544025a9-c3ef-4b22-b2d9-65be5a176dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.513295 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544025a9-c3ef-4b22-b2d9-65be5a176dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.513570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544025a9-c3ef-4b22-b2d9-65be5a176dc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.524843 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544025a9-c3ef-4b22-b2d9-65be5a176dc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-544025a9-c3ef-4b22-b2d9-65be5a176dc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:06.532439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:06.533516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:06.534348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 402/798] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:23:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:06.535897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee9546-11ce-4f4b-823d-3335d4f9aff2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.535897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee9546-11ce-4f4b-823d-3335d4f9aff2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.535897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee9546-11ce-4f4b-823d-3335d4f9aff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.554177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a7ee9546-11ce-4f4b-823d-3335d4f9aff2 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:06.556062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a7ee9546-11ce-4f4b-823d-3335d4f9aff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:06.557969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298fbcb-269b-476c-8bea-e543c366164b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:06.595380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505654d-7ab6-42a7-a126-fa41004ac827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:06.596502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505654d-7ab6-42a7-a126-fa41004ac827 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.596881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505654d-7ab6-42a7-a126-fa41004ac827 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.597156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505654d-7ab6-42a7-a126-fa41004ac827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.608653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505654d-7ab6-42a7-a126-fa41004ac827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:06.610110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505654d-7ab6-42a7-a126-fa41004ac827 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5505654d-7ab6-42a7-a126-fa41004ac827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:06.628258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:06.629167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:06.635251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:06.635251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:06.635251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:06.635251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:06.635251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 397/799] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:06.635251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505654d-7ab6-42a7-a126-fa41004ac827 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:06.638836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 403/800] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:06.639957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee9546-11ce-4f4b-823d-3335d4f9aff2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:06.643145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:06.698675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:06.747843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505654d-7ab6-42a7-a126-fa41004ac827 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:06.758396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee9546-11ce-4f4b-823d-3335d4f9aff2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:06.783126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40912bab-c073-4830-9b7b-d770947229d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:06.783979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40912bab-c073-4830-9b7b-d770947229d8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.784262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40912bab-c073-4830-9b7b-d770947229d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.784538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40912bab-c073-4830-9b7b-d770947229d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.793885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40912bab-c073-4830-9b7b-d770947229d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:06.794807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f91879-42c6-4b55-b396-c55b3de7187b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:06.795377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40912bab-c073-4830-9b7b-d770947229d8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:06.795482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f91879-42c6-4b55-b396-c55b3de7187b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.795748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f91879-42c6-4b55-b396-c55b3de7187b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.796018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f91879-42c6-4b55-b396-c55b3de7187b None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.796769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f91879-42c6-4b55-b396-c55b3de7187b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:06.797076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f91879-42c6-4b55-b396-c55b3de7187b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:06.798432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f91879-42c6-4b55-b396-c55b3de7187b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:06.806928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 404/801] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:23:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:06.807880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f91879-42c6-4b55-b396-c55b3de7187b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-40912bab-c073-4830-9b7b-d770947229d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:06.811932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:06.812861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:06.813787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:06.813787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:06.813787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:06.813787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:06.813787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:06.813787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:06.813787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:06.813787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:06.813787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:06.813787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:06.813787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 398/802] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jun 29 08:23:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:06.820154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40912bab-c073-4830-9b7b-d770947229d8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:06.878037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:06.917417 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f91879-42c6-4b55-b396-c55b3de7187b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:06.919958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40912bab-c073-4830-9b7b-d770947229d8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:06.933235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:06.951740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953f7d0-f373-43c2-8bfe-6b1f053eaaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:06.952341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953f7d0-f373-43c2-8bfe-6b1f053eaaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.952631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953f7d0-f373-43c2-8bfe-6b1f053eaaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.952911 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953f7d0-f373-43c2-8bfe-6b1f053eaaf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.953116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a401-4bdd-4e6d-9a72-7c2bfb3d0121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:06.953647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953f7d0-f373-43c2-8bfe-6b1f053eaaf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:06.953921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a401-4bdd-4e6d-9a72-7c2bfb3d0121 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:06.954016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953f7d0-f373-43c2-8bfe-6b1f053eaaf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:06.954200 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a401-4bdd-4e6d-9a72-7c2bfb3d0121 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:06.954469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570a401-4bdd-4e6d-9a72-7c2bfb3d0121 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:06.955296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570a401-4bdd-4e6d-9a72-7c2bfb3d0121 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:06.955770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953f7d0-f373-43c2-8bfe-6b1f053eaaf1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:06.956586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570a401-4bdd-4e6d-9a72-7c2bfb3d0121 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:06.967929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 405/803] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:06.968779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953f7d0-f373-43c2-8bfe-6b1f053eaaf1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f570a401-4bdd-4e6d-9a72-7c2bfb3d0121 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:06.971689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:06.972652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:06.978386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 399/804] 10.222.0.27 () {56 vars in 1129 bytes} [Wed Jun 29 08:23:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:06.979283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a401-4bdd-4e6d-9a72-7c2bfb3d0121 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.073489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953f7d0-f373-43c2-8bfe-6b1f053eaaf1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:07.074026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570a401-4bdd-4e6d-9a72-7c2bfb3d0121 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:07.098148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:07.109097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429224b0-7584-4fbd-8890-f2514ca2dab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:07.109284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a8ebc-4071-4973-854a-1f971b41344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:07.109895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429224b0-7584-4fbd-8890-f2514ca2dab0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.110175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429224b0-7584-4fbd-8890-f2514ca2dab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.110446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429224b0-7584-4fbd-8890-f2514ca2dab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.110536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a8ebc-4071-4973-854a-1f971b41344a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.110789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a8ebc-4071-4973-854a-1f971b41344a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.111069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a8ebc-4071-4973-854a-1f971b41344a None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.112227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a8ebc-4071-4973-854a-1f971b41344a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:07.112545 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a8ebc-4071-4973-854a-1f971b41344a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:07.113983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a8ebc-4071-4973-854a-1f971b41344a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:07.120883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429224b0-7584-4fbd-8890-f2514ca2dab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:07.122302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429224b0-7584-4fbd-8890-f2514ca2dab0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:07.131649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 400/805] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jun 29 08:23:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:07.132934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe281a-a502-42c1-9e87-3072db02afde None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.133225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe281a-a502-42c1-9e87-3072db02afde None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.133503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe281a-a502-42c1-9e87-3072db02afde None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.139033 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 401/806] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:07.139033 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe281a-a502-42c1-9e87-3072db02afde None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-429224b0-7584-4fbd-8890-f2514ca2dab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:07.141590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:07.142561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:07.143501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:07.143501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:07.143501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:07.143501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:07.143501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 406/807] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:07.144062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429224b0-7584-4fbd-8890-f2514ca2dab0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.172396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:07.238436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe281a-a502-42c1-9e87-3072db02afde None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:07.246284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429224b0-7584-4fbd-8890-f2514ca2dab0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:07.272702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25728a78-ab71-45e3-bd41-8137e197733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:07.273505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25728a78-ab71-45e3-bd41-8137e197733c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.273800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25728a78-ab71-45e3-bd41-8137e197733c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.274067 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25728a78-ab71-45e3-bd41-8137e197733c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.280871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25728a78-ab71-45e3-bd41-8137e197733c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:07.281960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16916fa-271d-456a-b196-c4d21f868a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:07.282464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25728a78-ab71-45e3-bd41-8137e197733c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:07.282598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16916fa-271d-456a-b196-c4d21f868a58 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.282923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16916fa-271d-456a-b196-c4d21f868a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.283270 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16916fa-271d-456a-b196-c4d21f868a58 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.283997 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16916fa-271d-456a-b196-c4d21f868a58 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:07.284291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16916fa-271d-456a-b196-c4d21f868a58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:07.285622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16916fa-271d-456a-b196-c4d21f868a58 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:07.295609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 407/808] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jun 29 08:23:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:07.295609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bbd6-ece9-4c05-8182-5bbcfd035c2b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.295609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bbd6-ece9-4c05-8182-5bbcfd035c2b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.295609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bbd6-ece9-4c05-8182-5bbcfd035c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-25728a78-ab71-45e3-bd41-8137e197733c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:07.296878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:07.297713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:07.298653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:07.298653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:07.298653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:07.298653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:07.298653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:07.298653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:07.298653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:07.298653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:07.298653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 402/809] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:07.299149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25728a78-ab71-45e3-bd41-8137e197733c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.317643 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0e91bbd6-ece9-4c05-8182-5bbcfd035c2b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:07.318907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0e91bbd6-ece9-4c05-8182-5bbcfd035c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:07.353053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:07.385344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 408/810] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:07.386160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91bbd6-ece9-4c05-8182-5bbcfd035c2b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.395930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:07.403085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25728a78-ab71-45e3-bd41-8137e197733c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:07.436720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93621674-d853-4163-a83d-81129ec0cea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:07.437515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93621674-d853-4163-a83d-81129ec0cea1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.437794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93621674-d853-4163-a83d-81129ec0cea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.438065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93621674-d853-4163-a83d-81129ec0cea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.449231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93621674-d853-4163-a83d-81129ec0cea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:07.450669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93621674-d853-4163-a83d-81129ec0cea1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-93621674-d853-4163-a83d-81129ec0cea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:07.469136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:07.470030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:07.471021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:07.471021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:07.471021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:07.471021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:07.471021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 403/811] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:07.471522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93621674-d853-4163-a83d-81129ec0cea1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.495283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91bbd6-ece9-4c05-8182-5bbcfd035c2b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:07.528775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df8146-b96a-4491-b516-b2392e21ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:07.529708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8146-b96a-4491-b516-b2392e21ab66 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.529988 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8146-b96a-4491-b516-b2392e21ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.530235 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df8146-b96a-4491-b516-b2392e21ab66 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.530979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df8146-b96a-4491-b516-b2392e21ab66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:07.531288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df8146-b96a-4491-b516-b2392e21ab66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:07.532658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df8146-b96a-4491-b516-b2392e21ab66 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:07.543947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 409/812] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:07.544509 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df8146-b96a-4491-b516-b2392e21ab66 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.575295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93621674-d853-4163-a83d-81129ec0cea1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:07.576167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:07.608336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c12cc-c8f5-4444-b81d-594bcd9c220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:07.609001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c12cc-c8f5-4444-b81d-594bcd9c220a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.609279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c12cc-c8f5-4444-b81d-594bcd9c220a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.609556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c12cc-c8f5-4444-b81d-594bcd9c220a None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.610261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c12cc-c8f5-4444-b81d-594bcd9c220a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:07.610567 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c12cc-c8f5-4444-b81d-594bcd9c220a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:07.612043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c12cc-c8f5-4444-b81d-594bcd9c220a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:07.616835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 404/813] 10.222.0.27 () {56 vars in 1124 bytes} [Wed Jun 29 08:23:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:07.617594 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c12cc-c8f5-4444-b81d-594bcd9c220a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.632489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:07.651928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df8146-b96a-4491-b516-b2392e21ab66 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:07.687707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1361d83-2cf2-4d59-b19b-be5c7d70340f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:07.688367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1361d83-2cf2-4d59-b19b-be5c7d70340f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.688644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1361d83-2cf2-4d59-b19b-be5c7d70340f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.688911 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1361d83-2cf2-4d59-b19b-be5c7d70340f None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.689740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1361d83-2cf2-4d59-b19b-be5c7d70340f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:07.690193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1361d83-2cf2-4d59-b19b-be5c7d70340f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:07.691664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1361d83-2cf2-4d59-b19b-be5c7d70340f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:07.703991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 410/814] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:07.705188 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1361d83-2cf2-4d59-b19b-be5c7d70340f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.725733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c12cc-c8f5-4444-b81d-594bcd9c220a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:07.759908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66884aa1-2f61-4dd7-9ae9-9e2b0e9bd3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:07.760612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66884aa1-2f61-4dd7-9ae9-9e2b0e9bd3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.760890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66884aa1-2f61-4dd7-9ae9-9e2b0e9bd3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.761160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66884aa1-2f61-4dd7-9ae9-9e2b0e9bd3c8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.761812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66884aa1-2f61-4dd7-9ae9-9e2b0e9bd3c8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:07.763216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66884aa1-2f61-4dd7-9ae9-9e2b0e9bd3c8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-66884aa1-2f61-4dd7-9ae9-9e2b0e9bd3c8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:07.774652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:07.775660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:07.776474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 405/815] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:07 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:07.777306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66884aa1-2f61-4dd7-9ae9-9e2b0e9bd3c8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.804242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:07.814442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1361d83-2cf2-4d59-b19b-be5c7d70340f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:07.851422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8666c-3fe9-4293-909c-aff8ddf912fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:07.852082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8666c-3fe9-4293-909c-aff8ddf912fe None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.852362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8666c-3fe9-4293-909c-aff8ddf912fe None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.852621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf8666c-3fe9-4293-909c-aff8ddf912fe None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.853361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8666c-3fe9-4293-909c-aff8ddf912fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:07.853667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8666c-3fe9-4293-909c-aff8ddf912fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:07.853878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:07.855109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf8666c-3fe9-4293-909c-aff8ddf912fe None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:07.868337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 411/816] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:07.869116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8666c-3fe9-4293-909c-aff8ddf912fe None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.879520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66884aa1-2f61-4dd7-9ae9-9e2b0e9bd3c8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:07.915069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa89714-f292-413e-ac80-1ade3da23533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:07.915869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa89714-f292-413e-ac80-1ade3da23533 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:07.916149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa89714-f292-413e-ac80-1ade3da23533 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:07.916427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa89714-f292-413e-ac80-1ade3da23533 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:07.924040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa89714-f292-413e-ac80-1ade3da23533 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:07.925813 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa89714-f292-413e-ac80-1ade3da23533 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-aaa89714-f292-413e-ac80-1ade3da23533 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:07.940127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:07.941066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 406/817] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jun 29 08:23:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:07.942496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa89714-f292-413e-ac80-1ade3da23533 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:07.970113 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf8666c-3fe9-4293-909c-aff8ddf912fe None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:08.003046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab12d1-2d52-4b2e-9dd7-d40220550368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:08.003725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab12d1-2d52-4b2e-9dd7-d40220550368 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:08.003995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab12d1-2d52-4b2e-9dd7-d40220550368 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:08.004267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab12d1-2d52-4b2e-9dd7-d40220550368 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:08.005012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab12d1-2d52-4b2e-9dd7-d40220550368 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:08.006393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab12d1-2d52-4b2e-9dd7-d40220550368 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:08.028832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 412/818] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jun 29 08:23:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:08.029583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab12d1-2d52-4b2e-9dd7-d40220550368 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:08.046854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:08.052064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa89714-f292-413e-ac80-1ade3da23533 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:08.095079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:08.101011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4c077-cf66-4a83-9f18-f2b83160bec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:08.101942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4c077-cf66-4a83-9f18-f2b83160bec5 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:08.102226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4c077-cf66-4a83-9f18-f2b83160bec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:08.102507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4c077-cf66-4a83-9f18-f2b83160bec5 None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/3b38411347754c058eb6640829078e1d/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:08.104725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4c077-cf66-4a83-9f18-f2b83160bec5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=3b38411347754c058eb6640829078e1d, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:08.106199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4c077-cf66-4a83-9f18-f2b83160bec5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:08.130724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 407/819] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:23:07 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/3b38411347754c058eb6640829078e1d/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:23:08.130724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4c077-cf66-4a83-9f18-f2b83160bec5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:08.173534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab12d1-2d52-4b2e-9dd7-d40220550368 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:08.227585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c974b4-d9ae-4d81-834d-b5de7e7a0645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:08.228291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c974b4-d9ae-4d81-834d-b5de7e7a0645 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:08.228619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c974b4-d9ae-4d81-834d-b5de7e7a0645 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:08.228893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c974b4-d9ae-4d81-834d-b5de7e7a0645 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:08.229661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c974b4-d9ae-4d81-834d-b5de7e7a0645 None admin] RBAC: Adding query filter params (name=network) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:08.229969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c974b4-d9ae-4d81-834d-b5de7e7a0645 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:08.231467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c974b4-d9ae-4d81-834d-b5de7e7a0645 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:08.238532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 413/820] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:23:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:08.239983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c974b4-d9ae-4d81-834d-b5de7e7a0645 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:08.267743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4c077-cf66-4a83-9f18-f2b83160bec5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:08.305265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271648-e32c-433f-bb09-ae40468f4cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:08.306466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271648-e32c-433f-bb09-ae40468f4cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:08.307008 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271648-e32c-433f-bb09-ae40468f4cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:08.307472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271648-e32c-433f-bb09-ae40468f4cd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:08.315926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75271648-e32c-433f-bb09-ae40468f4cd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:08.316823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75271648-e32c-433f-bb09-ae40468f4cd3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:08.331677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-75271648-e32c-433f-bb09-ae40468f4cd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:08.338192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:08.339580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:08.340439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:08.340439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:08.340439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:08.340439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:08.340439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:08.340439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:08.340439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:08.340439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:08.340858 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 408/821] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:08.341852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271648-e32c-433f-bb09-ae40468f4cd3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:08.379702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c974b4-d9ae-4d81-834d-b5de7e7a0645 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:08.410431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:08.416984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6139b5-5dd7-4cbc-8183-e2595cfd7464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:08.417671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6139b5-5dd7-4cbc-8183-e2595cfd7464 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:08.418104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6139b5-5dd7-4cbc-8183-e2595cfd7464 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:08.418381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6139b5-5dd7-4cbc-8183-e2595cfd7464 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:08.419272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6139b5-5dd7-4cbc-8183-e2595cfd7464 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:08.419586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6139b5-5dd7-4cbc-8183-e2595cfd7464 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:08.421035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6139b5-5dd7-4cbc-8183-e2595cfd7464 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:08.439963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 414/822] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jun 29 08:23:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:08.440936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6139b5-5dd7-4cbc-8183-e2595cfd7464 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:08.465796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75271648-e32c-433f-bb09-ae40468f4cd3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:08.504619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb8bac-c72f-4ce7-9ab7-fee45a8c1c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:08.505414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb8bac-c72f-4ce7-9ab7-fee45a8c1c9d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:08.505695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb8bac-c72f-4ce7-9ab7-fee45a8c1c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:08.505988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb8bac-c72f-4ce7-9ab7-fee45a8c1c9d None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:08.506756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb8bac-c72f-4ce7-9ab7-fee45a8c1c9d None admin] RBAC: Adding query filter params (type=network) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:08.507071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb8bac-c72f-4ce7-9ab7-fee45a8c1c9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:08.508562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb8bac-c72f-4ce7-9ab7-fee45a8c1c9d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:08.514385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 409/823] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:23:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:08.515228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb8bac-c72f-4ce7-9ab7-fee45a8c1c9d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:08.543768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6139b5-5dd7-4cbc-8183-e2595cfd7464 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:08.570968 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:08.579080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207af0c9-9f50-4b0d-84c9-a38d42bb87c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:08.579872 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207af0c9-9f50-4b0d-84c9-a38d42bb87c1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:08.580304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207af0c9-9f50-4b0d-84c9-a38d42bb87c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:08.580645 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207af0c9-9f50-4b0d-84c9-a38d42bb87c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:08.581523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207af0c9-9f50-4b0d-84c9-a38d42bb87c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:08.581905 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207af0c9-9f50-4b0d-84c9-a38d42bb87c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:08.583538 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207af0c9-9f50-4b0d-84c9-a38d42bb87c1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:08.601245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 415/824] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:08.602239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207af0c9-9f50-4b0d-84c9-a38d42bb87c1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:08.635226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb8bac-c72f-4ce7-9ab7-fee45a8c1c9d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:08.668063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:08.671874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6290d0e-4a48-4653-ace5-dbaeb2c6f480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:08.672538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6290d0e-4a48-4653-ace5-dbaeb2c6f480 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:08.672826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6290d0e-4a48-4653-ace5-dbaeb2c6f480 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:08.673105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6290d0e-4a48-4653-ace5-dbaeb2c6f480 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:08.706338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207af0c9-9f50-4b0d-84c9-a38d42bb87c1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:08.741284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407a893-9dae-47a6-9d54-302a21c7bc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:08.742095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407a893-9dae-47a6-9d54-302a21c7bc19 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:08.742472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407a893-9dae-47a6-9d54-302a21c7bc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:08.742874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407a893-9dae-47a6-9d54-302a21c7bc19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:08.743720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7407a893-9dae-47a6-9d54-302a21c7bc19 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:08.745199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7407a893-9dae-47a6-9d54-302a21c7bc19 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:08.747669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6290d0e-4a48-4653-ace5-dbaeb2c6f480 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:08.749257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6290d0e-4a48-4653-ace5-dbaeb2c6f480 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:08.772114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 416/825] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jun 29 08:23:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:08.773125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407a893-9dae-47a6-9d54-302a21c7bc19 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:08.833154 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:08.918139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407a893-9dae-47a6-9d54-302a21c7bc19 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:08.957116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 410/826] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:23:08.962454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6290d0e-4a48-4653-ace5-dbaeb2c6f480 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:08.964035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f63b7-95f6-45ae-88d9-10abc3f8b931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:08.965485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f63b7-95f6-45ae-88d9-10abc3f8b931 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:08.965970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f63b7-95f6-45ae-88d9-10abc3f8b931 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:08.966466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f63b7-95f6-45ae-88d9-10abc3f8b931 None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/f320b56bf7ff4c36bbd9730e688612d5/roles/8f88a54639c64acdbdb152911e5f9eb2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:08.984358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f63b7-95f6-45ae-88d9-10abc3f8b931 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=f320b56bf7ff4c36bbd9730e688612d5, role_id=8f88a54639c64acdbdb152911e5f9eb2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:08.987339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f63b7-95f6-45ae-88d9-10abc3f8b931 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:09.002931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:09.014815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 417/827] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:23:08 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/f320b56bf7ff4c36bbd9730e688612d5/roles/8f88a54639c64acdbdb152911e5f9eb2 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:23:09.134679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6290d0e-4a48-4653-ace5-dbaeb2c6f480 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:09.178011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ceb55-1327-4365-80ab-69bc84d9c48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:09.179000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ceb55-1327-4365-80ab-69bc84d9c48f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:09.179314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ceb55-1327-4365-80ab-69bc84d9c48f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:09.179582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11ceb55-1327-4365-80ab-69bc84d9c48f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:09.191330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ceb55-1327-4365-80ab-69bc84d9c48f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:09.192793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11ceb55-1327-4365-80ab-69bc84d9c48f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b11ceb55-1327-4365-80ab-69bc84d9c48f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:09.212575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:09.213881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:09.215012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:09.215012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:09.215012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:09.215012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:09.218788 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 411/828] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:09.221421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f63b7-95f6-45ae-88d9-10abc3f8b931 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:09.336455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:09.346672 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f63b7-95f6-45ae-88d9-10abc3f8b931 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:09.382667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a92ebe-e6e8-4863-9bb3-e6fb3a97d07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:09.383418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a92ebe-e6e8-4863-9bb3-e6fb3a97d07a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:09.383701 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a92ebe-e6e8-4863-9bb3-e6fb3a97d07a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:09.383975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a92ebe-e6e8-4863-9bb3-e6fb3a97d07a None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:09.384768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a92ebe-e6e8-4863-9bb3-e6fb3a97d07a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:09.385019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a92ebe-e6e8-4863-9bb3-e6fb3a97d07a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:09.386402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a92ebe-e6e8-4863-9bb3-e6fb3a97d07a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:09.398826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 418/829] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:09.406588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ceb55-1327-4365-80ab-69bc84d9c48f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:09.482148 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:09.516514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11ceb55-1327-4365-80ab-69bc84d9c48f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:09.553978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048db4b7-12f5-4c30-89bb-ca28a496dad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:09.554806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048db4b7-12f5-4c30-89bb-ca28a496dad4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:09.555089 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048db4b7-12f5-4c30-89bb-ca28a496dad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:09.555370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048db4b7-12f5-4c30-89bb-ca28a496dad4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:09.562296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048db4b7-12f5-4c30-89bb-ca28a496dad4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:09.563776 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048db4b7-12f5-4c30-89bb-ca28a496dad4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-048db4b7-12f5-4c30-89bb-ca28a496dad4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:09.577570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:09.578443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:09.579466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:09.579466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:09.579466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:09.579466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:09.579466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:09.579466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:09.579466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:09.579466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:09.579466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 412/830] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:09.584907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a92ebe-e6e8-4863-9bb3-e6fb3a97d07a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:09.687418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a92ebe-e6e8-4863-9bb3-e6fb3a97d07a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:09.722227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7f2bd-75b0-428b-803a-8b15bc5a015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:09.722924 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7f2bd-75b0-428b-803a-8b15bc5a015f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:09.723209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7f2bd-75b0-428b-803a-8b15bc5a015f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:09.723484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7f2bd-75b0-428b-803a-8b15bc5a015f None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:09.724275 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7f2bd-75b0-428b-803a-8b15bc5a015f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:09.724587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7f2bd-75b0-428b-803a-8b15bc5a015f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:09.726017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7f2bd-75b0-428b-803a-8b15bc5a015f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:09.742803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 419/831] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:09.744921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048db4b7-12f5-4c30-89bb-ca28a496dad4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:09.852500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048db4b7-12f5-4c30-89bb-ca28a496dad4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:09.883427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:09.889951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c8fc2-58fe-448d-b952-3a1b7cddba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:09.890818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c8fc2-58fe-448d-b952-3a1b7cddba9b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:09.891316 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c8fc2-58fe-448d-b952-3a1b7cddba9b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:09.891597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c8fc2-58fe-448d-b952-3a1b7cddba9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:09.892563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c8fc2-58fe-448d-b952-3a1b7cddba9b None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=7bd500f020f143339c5a4fc9f0728ee4) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:09.892886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c8fc2-58fe-448d-b952-3a1b7cddba9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:09.894384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c8fc2-58fe-448d-b952-3a1b7cddba9b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:09.901103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 413/832] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:09 2022] GET /identity/v3/role_assignments?user.id=7bd500f020f143339c5a4fc9f0728ee4&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:10.237825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa5b49-5b30-4a51-990b-1eee0142ada0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:10.238357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa5b49-5b30-4a51-990b-1eee0142ada0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:10.238357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa5b49-5b30-4a51-990b-1eee0142ada0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:10.240028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 420/833] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:10.245348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c1da9-a730-4b92-84db-ef7eb405827a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:10.245777 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c1da9-a730-4b92-84db-ef7eb405827a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:10.245956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c1da9-a730-4b92-84db-ef7eb405827a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:10.264037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d14c1da9-a730-4b92-84db-ef7eb405827a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:10.265248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d14c1da9-a730-4b92-84db-ef7eb405827a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:10.301010 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce5761-b28f-4cff-8275-46c9a1bd4e0b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:10.301487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce5761-b28f-4cff-8275-46c9a1bd4e0b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:10.301770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce5761-b28f-4cff-8275-46c9a1bd4e0b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:10.304324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 421/834] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:10.310267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5717bec-6230-4800-98b3-5ffb1d19c434 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:10.310558 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5717bec-6230-4800-98b3-5ffb1d19c434 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:10.310880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5717bec-6230-4800-98b3-5ffb1d19c434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:10.331442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c5717bec-6230-4800-98b3-5ffb1d19c434 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:10.333271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c5717bec-6230-4800-98b3-5ffb1d19c434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:10.336607 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:10.337195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 414/835] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:10.361103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c1da9-a730-4b92-84db-ef7eb405827a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:10.417512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 422/836] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:23:10.440413 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5717bec-6230-4800-98b3-5ffb1d19c434 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:10.465306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c1da9-a730-4b92-84db-ef7eb405827a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:10.489432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:10.501809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bbae0-99ca-4d4b-9bde-61dfd6ac8d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:10.502773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bbae0-99ca-4d4b-9bde-61dfd6ac8d3a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:10.503061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bbae0-99ca-4d4b-9bde-61dfd6ac8d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:10.503333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bbae0-99ca-4d4b-9bde-61dfd6ac8d3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:10.513775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467bbae0-99ca-4d4b-9bde-61dfd6ac8d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-467bbae0-99ca-4d4b-9bde-61dfd6ac8d3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:10.521818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:10.524092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:10.526143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 415/837] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:23:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:10.530809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bbae0-99ca-4d4b-9bde-61dfd6ac8d3a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:10.544244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5717bec-6230-4800-98b3-5ffb1d19c434 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:10.580844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbab308-7d0a-44c7-9b64-5afe7b9bd328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:10.581709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbab308-7d0a-44c7-9b64-5afe7b9bd328 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:10.581996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbab308-7d0a-44c7-9b64-5afe7b9bd328 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:10.582278 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbab308-7d0a-44c7-9b64-5afe7b9bd328 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:10.582980 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbab308-7d0a-44c7-9b64-5afe7b9bd328 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:10.584489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbab308-7d0a-44c7-9b64-5afe7b9bd328 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3cbab308-7d0a-44c7-9b64-5afe7b9bd328 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:10.597057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:10.598254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 08:23:10.599194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:10.603171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 423/838] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:23:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:10.604037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbab308-7d0a-44c7-9b64-5afe7b9bd328 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:10.607728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:10.637515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bbae0-99ca-4d4b-9bde-61dfd6ac8d3a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:10.671922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2996bd3-df9b-4e0c-b32c-9406e612341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:10.672959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2996bd3-df9b-4e0c-b32c-9406e612341f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:10.673442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2996bd3-df9b-4e0c-b32c-9406e612341f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:10.674024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2996bd3-df9b-4e0c-b32c-9406e612341f None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:10.675042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2996bd3-df9b-4e0c-b32c-9406e612341f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:10.675518 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2996bd3-df9b-4e0c-b32c-9406e612341f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:10.677063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2996bd3-df9b-4e0c-b32c-9406e612341f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:10.690767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 416/839] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:23:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:10.691935 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2996bd3-df9b-4e0c-b32c-9406e612341f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:10.710545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbab308-7d0a-44c7-9b64-5afe7b9bd328 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:10.743157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:10.746924 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578867-999c-4d30-a16a-984cfa16667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:10.747609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578867-999c-4d30-a16a-984cfa16667a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:10.747890 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578867-999c-4d30-a16a-984cfa16667a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:10.748171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578867-999c-4d30-a16a-984cfa16667a None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:10.749434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e578867-999c-4d30-a16a-984cfa16667a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:10.749434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e578867-999c-4d30-a16a-984cfa16667a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:10.750719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e578867-999c-4d30-a16a-984cfa16667a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:10.760678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 424/840] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:23:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:10.761545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578867-999c-4d30-a16a-984cfa16667a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:10.796972 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2996bd3-df9b-4e0c-b32c-9406e612341f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:10.831361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:10.832085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a054c-4fc0-4594-bafb-4b13055e4dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:10.833145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a054c-4fc0-4594-bafb-4b13055e4dfa None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:10.833655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a054c-4fc0-4594-bafb-4b13055e4dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:10.834131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a054c-4fc0-4594-bafb-4b13055e4dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:10.846334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a054c-4fc0-4594-bafb-4b13055e4dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:10.848082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a054c-4fc0-4594-bafb-4b13055e4dfa None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-971a054c-4fc0-4594-bafb-4b13055e4dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:10.865751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:10.866663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:10.867644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:10.867644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:10.867644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:10.867644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:10.869633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578867-999c-4d30-a16a-984cfa16667a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:10.874423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 417/841] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:10.875491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a054c-4fc0-4594-bafb-4b13055e4dfa None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:10.907320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7edf1d-176c-4257-ad73-dd1c96459912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:10.908157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7edf1d-176c-4257-ad73-dd1c96459912 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:10.908635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7edf1d-176c-4257-ad73-dd1c96459912 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:10.908939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7edf1d-176c-4257-ad73-dd1c96459912 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:10.909721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7edf1d-176c-4257-ad73-dd1c96459912 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:10.909980 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7edf1d-176c-4257-ad73-dd1c96459912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:10.911394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7edf1d-176c-4257-ad73-dd1c96459912 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:10.922923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 425/842] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:23:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:10.981391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:10.987564 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a054c-4fc0-4594-bafb-4b13055e4dfa None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:11.025376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3116c-0d89-4b59-b055-0ce40c0eb26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:11.025971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3116c-0d89-4b59-b055-0ce40c0eb26c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:11.026249 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3116c-0d89-4b59-b055-0ce40c0eb26c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:11.026521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3116c-0d89-4b59-b055-0ce40c0eb26c None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:11.027287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3116c-0d89-4b59-b055-0ce40c0eb26c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:11.027593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3116c-0d89-4b59-b055-0ce40c0eb26c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:11.029377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3116c-0d89-4b59-b055-0ce40c0eb26c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:11.042932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 418/843] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:11.050777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7edf1d-176c-4257-ad73-dd1c96459912 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:11.163604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7edf1d-176c-4257-ad73-dd1c96459912 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:11.194840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:11.199705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87427b51-4fcd-4187-a708-4c5f58e002fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:11.200520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87427b51-4fcd-4187-a708-4c5f58e002fb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:11.201065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87427b51-4fcd-4187-a708-4c5f58e002fb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:11.201344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87427b51-4fcd-4187-a708-4c5f58e002fb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:11.209873 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87427b51-4fcd-4187-a708-4c5f58e002fb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:11.211379 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87427b51-4fcd-4187-a708-4c5f58e002fb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:11.222236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742d3fa-d995-4eaa-bb5f-c7e451dad852 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:11.222526 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742d3fa-d995-4eaa-bb5f-c7e451dad852 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:11.222847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742d3fa-d995-4eaa-bb5f-c7e451dad852 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:11.224485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 419/844] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-87427b51-4fcd-4187-a708-4c5f58e002fb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:11.226770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:11.227784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:11.228983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:11.228983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:11.228983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:11.228983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:11.228983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:11.228983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:11.228983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:11.228983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:11.228983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 08:23:11.228983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:11.228983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 426/845] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:23:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:11.229723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341dcf7-dafc-4072-a07e-b3b9b0b2bbae None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:11.229883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341dcf7-dafc-4072-a07e-b3b9b0b2bbae None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:11.230159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341dcf7-dafc-4072-a07e-b3b9b0b2bbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:11.233469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87427b51-4fcd-4187-a708-4c5f58e002fb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:11.249278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b341dcf7-dafc-4072-a07e-b3b9b0b2bbae None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:11.250469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b341dcf7-dafc-4072-a07e-b3b9b0b2bbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:11.322781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 420/846] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:11.347180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341dcf7-dafc-4072-a07e-b3b9b0b2bbae None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:11.355246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87427b51-4fcd-4187-a708-4c5f58e002fb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:11.389047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae7033-f440-456f-82f3-4bbccbd5da22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:11.389768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae7033-f440-456f-82f3-4bbccbd5da22 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:11.390052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae7033-f440-456f-82f3-4bbccbd5da22 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:11.390325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae7033-f440-456f-82f3-4bbccbd5da22 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:11.391220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae7033-f440-456f-82f3-4bbccbd5da22 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:11.391532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae7033-f440-456f-82f3-4bbccbd5da22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:11.393343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ae7033-f440-456f-82f3-4bbccbd5da22 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:11.411705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 427/847] 10.222.0.27 () {56 vars in 1135 bytes} [Wed Jun 29 08:23:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:11.418068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae7033-f440-456f-82f3-4bbccbd5da22 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:11.449923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:11.481100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341dcf7-dafc-4072-a07e-b3b9b0b2bbae None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:11.490754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:11.523817 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e45787-7599-448e-a375-5585ac9b0499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:11.524631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e45787-7599-448e-a375-5585ac9b0499 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:11.524922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e45787-7599-448e-a375-5585ac9b0499 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:11.525192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e45787-7599-448e-a375-5585ac9b0499 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:11.536276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e45787-7599-448e-a375-5585ac9b0499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b4e45787-7599-448e-a375-5585ac9b0499 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:11.544243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:11.545160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:11.546207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 421/848] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:23:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:11.550781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae7033-f440-456f-82f3-4bbccbd5da22 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:11.551295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e45787-7599-448e-a375-5585ac9b0499 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:11.595643 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c9ec2-f41f-4fbb-908c-8d688eaa97bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:11.597468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c9ec2-f41f-4fbb-908c-8d688eaa97bd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:11.597763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c9ec2-f41f-4fbb-908c-8d688eaa97bd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:11.598048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c9ec2-f41f-4fbb-908c-8d688eaa97bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:11.684532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e45787-7599-448e-a375-5585ac9b0499 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:11.689423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c9ec2-f41f-4fbb-908c-8d688eaa97bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:11.691196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c9ec2-f41f-4fbb-908c-8d688eaa97bd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:11.728985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc0e8f-4b9d-4220-a2e3-6a1bb27fa196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:11.730150 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc0e8f-4b9d-4220-a2e3-6a1bb27fa196 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:11.730582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc0e8f-4b9d-4220-a2e3-6a1bb27fa196 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:11.731037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc0e8f-4b9d-4220-a2e3-6a1bb27fa196 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:11.732174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc0e8f-4b9d-4220-a2e3-6a1bb27fa196 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:11.733523 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc0e8f-4b9d-4220-a2e3-6a1bb27fa196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:11.735136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc0e8f-4b9d-4220-a2e3-6a1bb27fa196 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:11.750788 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 422/849] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:23:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:11.751920 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc0e8f-4b9d-4220-a2e3-6a1bb27fa196 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:11.756555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:11.768306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:11.825219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 428/850] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:11 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:23:11.834494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c9ec2-f41f-4fbb-908c-8d688eaa97bd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:11.878808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc0e8f-4b9d-4220-a2e3-6a1bb27fa196 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:11.924379 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f259df-88d9-4a77-8110-ff4ce8964823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:11.925478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f259df-88d9-4a77-8110-ff4ce8964823 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:11.927080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f259df-88d9-4a77-8110-ff4ce8964823 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:11.927456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f259df-88d9-4a77-8110-ff4ce8964823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:11.942582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f259df-88d9-4a77-8110-ff4ce8964823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:11.944869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f259df-88d9-4a77-8110-ff4ce8964823 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:11.961916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c9ec2-f41f-4fbb-908c-8d688eaa97bd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-82f259df-88d9-4a77-8110-ff4ce8964823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:11.967556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:11.968607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:11.969499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:11.969499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:11.969499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:11.969499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:11.970025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 423/851] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:11.975148 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f259df-88d9-4a77-8110-ff4ce8964823 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:12.019965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a21972-5127-4fb7-9132-cd5921b32aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:12.020829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21972-5127-4fb7-9132-cd5921b32aef None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.021528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21972-5127-4fb7-9132-cd5921b32aef None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.021812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21972-5127-4fb7-9132-cd5921b32aef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.035492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a21972-5127-4fb7-9132-cd5921b32aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:12.036885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a21972-5127-4fb7-9132-cd5921b32aef None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:12.059125 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c6a21972-5127-4fb7-9132-cd5921b32aef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:12.059453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:12.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:12.061147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:12.061147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:12.061147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:12.061147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:12.061147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 429/852] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:12.066309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a21972-5127-4fb7-9132-cd5921b32aef None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:12.069510 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:12.113461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f259df-88d9-4a77-8110-ff4ce8964823 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:12.151864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e3d28-011f-431a-9627-7c19d65ba2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:12.152453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e3d28-011f-431a-9627-7c19d65ba2be None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.152889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e3d28-011f-431a-9627-7c19d65ba2be None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.153177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e3d28-011f-431a-9627-7c19d65ba2be None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.153921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e3d28-011f-431a-9627-7c19d65ba2be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:12.154237 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e3d28-011f-431a-9627-7c19d65ba2be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:12.155661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e3d28-011f-431a-9627-7c19d65ba2be None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:12.172961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 424/853] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:12.173780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a21972-5127-4fb7-9132-cd5921b32aef None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:12.174544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68246d-5609-4616-b87f-7d0f9f18c13a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.174865 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68246d-5609-4616-b87f-7d0f9f18c13a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.175200 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68246d-5609-4616-b87f-7d0f9f18c13a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.176825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 425/854] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:12.177368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68246d-5609-4616-b87f-7d0f9f18c13a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:12.211410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa32568-36bc-4bf2-896a-e5b86a78d8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:12.212076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa32568-36bc-4bf2-896a-e5b86a78d8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.212384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa32568-36bc-4bf2-896a-e5b86a78d8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.212628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa32568-36bc-4bf2-896a-e5b86a78d8e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.213690 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa32568-36bc-4bf2-896a-e5b86a78d8e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:12.214054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa32568-36bc-4bf2-896a-e5b86a78d8e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:12.215404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa32568-36bc-4bf2-896a-e5b86a78d8e3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:12.232900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 430/855] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:12.232900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8e7f-001c-4108-86b8-9e2e2416ae82 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.232900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8e7f-001c-4108-86b8-9e2e2416ae82 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.232900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d8e7f-001c-4108-86b8-9e2e2416ae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.248156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-677d8e7f-001c-4108-86b8-9e2e2416ae82 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:12.249321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-677d8e7f-001c-4108-86b8-9e2e2416ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:12.289709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68246d-5609-4616-b87f-7d0f9f18c13a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:12.307606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:12.315368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:12.322590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 431/856] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:12.323844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d8e7f-001c-4108-86b8-9e2e2416ae82 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:12.330379 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c7691-e218-4cd4-937c-1ff573a5205d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:12.331508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c7691-e218-4cd4-937c-1ff573a5205d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.331884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c7691-e218-4cd4-937c-1ff573a5205d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.332248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c7691-e218-4cd4-937c-1ff573a5205d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.341294 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c7691-e218-4cd4-937c-1ff573a5205d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:12.342890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c7691-e218-4cd4-937c-1ff573a5205d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-826c7691-e218-4cd4-937c-1ff573a5205d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:12.359099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:12.360240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:12.361579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:12.361579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:12.361579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:12.361579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:12.361579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:12.361579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:12.361579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:12.361579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:12.361579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:12.361579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:12.361579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 426/857] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jun 29 08:23:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:12.362296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c7691-e218-4cd4-937c-1ff573a5205d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:12.447268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d8e7f-001c-4108-86b8-9e2e2416ae82 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:12.471879 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c7691-e218-4cd4-937c-1ff573a5205d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:12.488019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e0b45-d9c0-4e86-8819-67526452bed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:12.488823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e0b45-d9c0-4e86-8819-67526452bed8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.489107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e0b45-d9c0-4e86-8819-67526452bed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.489443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e0b45-d9c0-4e86-8819-67526452bed8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.496077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e0b45-d9c0-4e86-8819-67526452bed8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:12.497534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e0b45-d9c0-4e86-8819-67526452bed8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:12.507683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacdeca-d459-407a-912b-256366db8801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:12.508625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacdeca-d459-407a-912b-256366db8801 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.509051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacdeca-d459-407a-912b-256366db8801 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.509423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacdeca-d459-407a-912b-256366db8801 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.510229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacdeca-d459-407a-912b-256366db8801 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-081e0b45-d9c0-4e86-8819-67526452bed8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:12.511330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:12.512358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:12.513447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:12.513447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:12.513447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:12.513447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:12.513447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:12.513447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:12.513447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:12.513447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:12.514123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacdeca-d459-407a-912b-256366db8801 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:12.517509 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 432/858] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:12.518216 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e0b45-d9c0-4e86-8819-67526452bed8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-eeacdeca-d459-407a-912b-256366db8801 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:12.527647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:12.528680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:12.533505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 427/859] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Jun 29 08:23:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:12.534481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacdeca-d459-407a-912b-256366db8801 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:12.579425 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:12.582739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:12.630817 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e0b45-d9c0-4e86-8819-67526452bed8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:12.640579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacdeca-d459-407a-912b-256366db8801 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:12.668211 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd9b41-3024-4758-a997-9208b9632d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:12.669188 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd9b41-3024-4758-a997-9208b9632d12 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.669476 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd9b41-3024-4758-a997-9208b9632d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.670012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd9b41-3024-4758-a997-9208b9632d12 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.671079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd9b41-3024-4758-a997-9208b9632d12 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:12.671403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd9b41-3024-4758-a997-9208b9632d12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:12.673421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd9b41-3024-4758-a997-9208b9632d12 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:12.676757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdfb5f-e692-4e16-9ffe-b06d2a6f213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:12.677430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdfb5f-e692-4e16-9ffe-b06d2a6f213b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.677708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdfb5f-e692-4e16-9ffe-b06d2a6f213b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.677978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdfb5f-e692-4e16-9ffe-b06d2a6f213b None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.678842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdfb5f-e692-4e16-9ffe-b06d2a6f213b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:12.679151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdfb5f-e692-4e16-9ffe-b06d2a6f213b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:12.680571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bdfb5f-e692-4e16-9ffe-b06d2a6f213b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:12.692850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 428/860] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:12.693658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdfb5f-e692-4e16-9ffe-b06d2a6f213b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:12.695174 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 433/861] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jun 29 08:23:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:12.703344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd9b41-3024-4758-a997-9208b9632d12 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:12.807439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bdfb5f-e692-4e16-9ffe-b06d2a6f213b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:12.810661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd9b41-3024-4758-a997-9208b9632d12 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:12.810882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:12.822133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:12.846812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de975ffe-c531-4c3e-9692-c67a5a8a96a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:12.847483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de975ffe-c531-4c3e-9692-c67a5a8a96a6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.847766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de975ffe-c531-4c3e-9692-c67a5a8a96a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.847863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312b05e-b0ab-4372-92a7-91651779027a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:12.848059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de975ffe-c531-4c3e-9692-c67a5a8a96a6 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.848480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312b05e-b0ab-4372-92a7-91651779027a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:12.848743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de975ffe-c531-4c3e-9692-c67a5a8a96a6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:12.848836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312b05e-b0ab-4372-92a7-91651779027a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:12.849028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312b05e-b0ab-4372-92a7-91651779027a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:12.849130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de975ffe-c531-4c3e-9692-c67a5a8a96a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:12.849874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312b05e-b0ab-4372-92a7-91651779027a None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=3b38411347754c058eb6640829078e1d) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:12.850190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312b05e-b0ab-4372-92a7-91651779027a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:12.850409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de975ffe-c531-4c3e-9692-c67a5a8a96a6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:12.851655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312b05e-b0ab-4372-92a7-91651779027a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:12.856045 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 429/862] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jun 29 08:23:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:12.856849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de975ffe-c531-4c3e-9692-c67a5a8a96a6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:12.867036 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 434/863] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:12 2022] GET /identity/v3/role_assignments?user.id=3b38411347754c058eb6640829078e1d&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:12.868711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312b05e-b0ab-4372-92a7-91651779027a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:12.978234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de975ffe-c531-4c3e-9692-c67a5a8a96a6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:12.989179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312b05e-b0ab-4372-92a7-91651779027a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:13.019897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c5c66-f656-458b-bb8d-92a212848c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:13.021048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c5c66-f656-458b-bb8d-92a212848c38 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:13.021532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c5c66-f656-458b-bb8d-92a212848c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:13.022018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c5c66-f656-458b-bb8d-92a212848c38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:13.025935 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc9214-7f48-4e4c-a6ae-434ec7fb5352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:13.026758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc9214-7f48-4e4c-a6ae-434ec7fb5352 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:13.027306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc9214-7f48-4e4c-a6ae-434ec7fb5352 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:13.027583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc9214-7f48-4e4c-a6ae-434ec7fb5352 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:13.028942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc9214-7f48-4e4c-a6ae-434ec7fb5352 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:13.028942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc9214-7f48-4e4c-a6ae-434ec7fb5352 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:13.029989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc9214-7f48-4e4c-a6ae-434ec7fb5352 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:13.037345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 435/864] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jun 29 08:23:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:13.038380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba25af6-7eb8-4e42-99c1-350d08205821 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:13.038676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba25af6-7eb8-4e42-99c1-350d08205821 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:13.039160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba25af6-7eb8-4e42-99c1-350d08205821 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:13.040484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 436/865] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:13.046217 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3cc45-e105-490e-9eb6-8c432b81d907 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:13.046504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3cc45-e105-490e-9eb6-8c432b81d907 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:13.046842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3cc45-e105-490e-9eb6-8c432b81d907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:13.066447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9dc3cc45-e105-490e-9eb6-8c432b81d907 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:13.067475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:13.067876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9dc3cc45-e105-490e-9eb6-8c432b81d907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:13.110547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:13.119869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c5c66-f656-458b-bb8d-92a212848c38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:13.121373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c5c66-f656-458b-bb8d-92a212848c38 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:13.164812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 437/866] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:23:13.189537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3cc45-e105-490e-9eb6-8c432b81d907 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:13.250017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 430/867] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:12 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:23:13.256610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c5c66-f656-458b-bb8d-92a212848c38 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:13.302943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3cc45-e105-490e-9eb6-8c432b81d907 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:13.325632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:13.339557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5bf0f-3aff-4475-8436-2e55a8606ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:13.340237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5bf0f-3aff-4475-8436-2e55a8606ebd None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:13.340523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5bf0f-3aff-4475-8436-2e55a8606ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:13.340802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5bf0f-3aff-4475-8436-2e55a8606ebd None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:13.341543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5bf0f-3aff-4475-8436-2e55a8606ebd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:13.342963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5bf0f-3aff-4475-8436-2e55a8606ebd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:13.359492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 438/868] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:23:13 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:23:13.371629 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c5c66-f656-458b-bb8d-92a212848c38 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:13.423432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f4c45-93ba-4689-b7ed-e1ffa0d54f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:13.424995 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f4c45-93ba-4689-b7ed-e1ffa0d54f2d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:13.425509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f4c45-93ba-4689-b7ed-e1ffa0d54f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:13.425996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f4c45-93ba-4689-b7ed-e1ffa0d54f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:13.440999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f4c45-93ba-4689-b7ed-e1ffa0d54f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:13.446215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f4c45-93ba-4689-b7ed-e1ffa0d54f2d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-7a8f4c45-93ba-4689-b7ed-e1ffa0d54f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:13.473004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:13.473959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:13.474864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:13.474864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:13.474864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:13.474864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:13.477259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 431/869] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:13.482786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5bf0f-3aff-4475-8436-2e55a8606ebd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:13.547754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:13.593697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5bf0f-3aff-4475-8436-2e55a8606ebd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:13.628761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02ca3-9e2d-4478-8bc0-890fcda12e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:13.629442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02ca3-9e2d-4478-8bc0-890fcda12e6e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:13.629727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02ca3-9e2d-4478-8bc0-890fcda12e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:13.630002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02ca3-9e2d-4478-8bc0-890fcda12e6e None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:13.630761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe02ca3-9e2d-4478-8bc0-890fcda12e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:13.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe02ca3-9e2d-4478-8bc0-890fcda12e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:13.632442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe02ca3-9e2d-4478-8bc0-890fcda12e6e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:13.645029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 439/870] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:13.650485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f4c45-93ba-4689-b7ed-e1ffa0d54f2d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:13.776180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f4c45-93ba-4689-b7ed-e1ffa0d54f2d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:13.803218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:13.811610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf256537-9269-4dd8-8f63-a8c6a70f1c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:13.812679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256537-9269-4dd8-8f63-a8c6a70f1c53 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:13.813165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256537-9269-4dd8-8f63-a8c6a70f1c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:13.813628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256537-9269-4dd8-8f63-a8c6a70f1c53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:13.821055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf256537-9269-4dd8-8f63-a8c6a70f1c53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:13.822779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf256537-9269-4dd8-8f63-a8c6a70f1c53 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-cf256537-9269-4dd8-8f63-a8c6a70f1c53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:13.837928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:13.839220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:13.840076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:13.840076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:13.840076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:13.840076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:13.840076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:13.840076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:13.840076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:13.840076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:13.842501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 432/871] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:13.848069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02ca3-9e2d-4478-8bc0-890fcda12e6e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:13.943861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:13.957074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02ca3-9e2d-4478-8bc0-890fcda12e6e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:13.992395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c109e-09ca-48dd-b95c-10b562d6a8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:13.993434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c109e-09ca-48dd-b95c-10b562d6a8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:13.993908 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c109e-09ca-48dd-b95c-10b562d6a8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:13.994395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c109e-09ca-48dd-b95c-10b562d6a8f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:13.995456 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c109e-09ca-48dd-b95c-10b562d6a8f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:13.996000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c109e-09ca-48dd-b95c-10b562d6a8f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:13.997631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c109e-09ca-48dd-b95c-10b562d6a8f7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:14.010879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 440/872] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:14.019812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf256537-9269-4dd8-8f63-a8c6a70f1c53 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:14.127187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf256537-9269-4dd8-8f63-a8c6a70f1c53 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:14.161804 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabff13-2a90-499c-98a4-9ab1241915d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:14.162471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabff13-2a90-499c-98a4-9ab1241915d9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:14.162784 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabff13-2a90-499c-98a4-9ab1241915d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:14.163060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabff13-2a90-499c-98a4-9ab1241915d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:14.163900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabff13-2a90-499c-98a4-9ab1241915d9 None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=f320b56bf7ff4c36bbd9730e688612d5) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:14.164222 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabff13-2a90-499c-98a4-9ab1241915d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:14.165669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabff13-2a90-499c-98a4-9ab1241915d9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:14.172279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 433/873] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:14 2022] GET /identity/v3/role_assignments?user.id=f320b56bf7ff4c36bbd9730e688612d5&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:15.030933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862672b8-a76a-41b8-8616-b685843279a0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:15.031475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862672b8-a76a-41b8-8616-b685843279a0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:15.031475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862672b8-a76a-41b8-8616-b685843279a0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:15.033241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 441/874] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:15.038910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c33afd-8158-49f7-b589-7e102279ff49 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:15.039331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c33afd-8158-49f7-b589-7e102279ff49 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:15.039537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c33afd-8158-49f7-b589-7e102279ff49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:15.056555 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f7c33afd-8158-49f7-b589-7e102279ff49 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:15.057761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f7c33afd-8158-49f7-b589-7e102279ff49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:15.093257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:15.112403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f7aee-af43-4101-b560-b9d818accc06 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:15.112688 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f7aee-af43-4101-b560-b9d818accc06 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:15.112966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713f7aee-af43-4101-b560-b9d818accc06 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:15.115725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 442/875] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:15.119785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071e49c-8a91-4e85-9293-2a21fe87c655 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:15.120079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071e49c-8a91-4e85-9293-2a21fe87c655 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:15.120362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071e49c-8a91-4e85-9293-2a21fe87c655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:15.127644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 434/876] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:15 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:15.137391 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8071e49c-8a91-4e85-9293-2a21fe87c655 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:15.138588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8071e49c-8a91-4e85-9293-2a21fe87c655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:15.150064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c33afd-8158-49f7-b589-7e102279ff49 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:15.229670 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 443/877] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:15 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:23:15.252805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071e49c-8a91-4e85-9293-2a21fe87c655 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:15.269728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c33afd-8158-49f7-b589-7e102279ff49 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:15.306753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5125c1-5dd8-4b5d-8213-deb40cac5d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:15.308010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5125c1-5dd8-4b5d-8213-deb40cac5d27 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:15.308500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5125c1-5dd8-4b5d-8213-deb40cac5d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:15.308968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5125c1-5dd8-4b5d-8213-deb40cac5d27 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:15.309827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5125c1-5dd8-4b5d-8213-deb40cac5d27 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:15.311695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5125c1-5dd8-4b5d-8213-deb40cac5d27 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3b5125c1-5dd8-4b5d-8213-deb40cac5d27 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:15.326309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:15.327346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:15.328165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:15.331125 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 435/878] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jun 29 08:23:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:15.337248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5125c1-5dd8-4b5d-8213-deb40cac5d27 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:15.351441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:15.381614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071e49c-8a91-4e85-9293-2a21fe87c655 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:15.383687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:15.424027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e55ce-e48d-46bd-b341-33a21e27bb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:15.425949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e55ce-e48d-46bd-b341-33a21e27bb55 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:15.426828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e55ce-e48d-46bd-b341-33a21e27bb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:15.427533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e55ce-e48d-46bd-b341-33a21e27bb55 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:15.428530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e55ce-e48d-46bd-b341-33a21e27bb55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:15.430341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e55ce-e48d-46bd-b341-33a21e27bb55 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:15.455205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 444/879] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:23:15 2022] POST /identity/v3/services => generated 269 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:15.476651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5125c1-5dd8-4b5d-8213-deb40cac5d27 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:15.521739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81cbd4-3567-438c-8d75-8eb7a195249b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:15.522536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81cbd4-3567-438c-8d75-8eb7a195249b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:15.522871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81cbd4-3567-438c-8d75-8eb7a195249b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:15.523207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81cbd4-3567-438c-8d75-8eb7a195249b None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:15.523948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81cbd4-3567-438c-8d75-8eb7a195249b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:15.524312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81cbd4-3567-438c-8d75-8eb7a195249b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:15.525915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81cbd4-3567-438c-8d75-8eb7a195249b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:15.532103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 436/880] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jun 29 08:23:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:15.538169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e55ce-e48d-46bd-b341-33a21e27bb55 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:15.565526 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457bf78-9809-41f7-959b-56e66190c1c5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:15.566115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457bf78-9809-41f7-959b-56e66190c1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:15.566589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457bf78-9809-41f7-959b-56e66190c1c5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:15.568588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 437/881] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:15.574139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b25dad-cdd6-49d5-853a-2efc3331f4ac None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:15.574414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b25dad-cdd6-49d5-853a-2efc3331f4ac None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:15.574712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b25dad-cdd6-49d5-853a-2efc3331f4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:15.592624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-34b25dad-cdd6-49d5-853a-2efc3331f4ac None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:15.593840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-34b25dad-cdd6-49d5-853a-2efc3331f4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:15.667518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e55ce-e48d-46bd-b341-33a21e27bb55 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:15.670255 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 438/882] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:23:15.693821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b25dad-cdd6-49d5-853a-2efc3331f4ac None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:15.715921 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607490-78f3-45a8-9887-5e43a5d3faa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:15.719177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607490-78f3-45a8-9887-5e43a5d3faa0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:15.719177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607490-78f3-45a8-9887-5e43a5d3faa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:15.719177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607490-78f3-45a8-9887-5e43a5d3faa0 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:15.719177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80607490-78f3-45a8-9887-5e43a5d3faa0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:15.719177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80607490-78f3-45a8-9887-5e43a5d3faa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:15.719794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80607490-78f3-45a8-9887-5e43a5d3faa0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:15.723856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:15.730724 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 445/883] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jun 29 08:23:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:15.815702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b25dad-cdd6-49d5-853a-2efc3331f4ac None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:15.854724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc183b-089f-4958-ba2f-40a693fc7521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:15.855565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc183b-089f-4958-ba2f-40a693fc7521 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:15.855857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc183b-089f-4958-ba2f-40a693fc7521 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:15.856130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc183b-089f-4958-ba2f-40a693fc7521 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:15.856809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc183b-089f-4958-ba2f-40a693fc7521 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:15.858233 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc183b-089f-4958-ba2f-40a693fc7521 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-21fc183b-089f-4958-ba2f-40a693fc7521 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:15.871218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:15.872284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:15.873778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 439/884] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:23:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:15.878725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607490-78f3-45a8-9887-5e43a5d3faa0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:15.881782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:15.990809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607490-78f3-45a8-9887-5e43a5d3faa0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:16.025731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aaae9-3bac-40bc-a9ca-073b6dc9cf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:16.026781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aaae9-3bac-40bc-a9ca-073b6dc9cf10 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:16.027260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aaae9-3bac-40bc-a9ca-073b6dc9cf10 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:16.027727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aaae9-3bac-40bc-a9ca-073b6dc9cf10 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:16.028635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aaae9-3bac-40bc-a9ca-073b6dc9cf10 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:16.029171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aaae9-3bac-40bc-a9ca-073b6dc9cf10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:16.030832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aaae9-3bac-40bc-a9ca-073b6dc9cf10 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:16.036844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 446/885] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:23:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:16.044937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc183b-089f-4958-ba2f-40a693fc7521 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:16.134422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:16.147005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc183b-089f-4958-ba2f-40a693fc7521 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:16.182020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b094d-d7c3-4012-841b-8317c8bd0c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:16.182716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b094d-d7c3-4012-841b-8317c8bd0c7a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:16.183013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b094d-d7c3-4012-841b-8317c8bd0c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:16.183398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b094d-d7c3-4012-841b-8317c8bd0c7a None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:16.184129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b094d-d7c3-4012-841b-8317c8bd0c7a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:16.184444 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b094d-d7c3-4012-841b-8317c8bd0c7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:16.185823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b094d-d7c3-4012-841b-8317c8bd0c7a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:16.191793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 440/886] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:23:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:16.197426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aaae9-3bac-40bc-a9ca-073b6dc9cf10 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:16.270977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:16.285243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096634af-7680-41c3-ae09-6aaf9eced1d1 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:16.285588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096634af-7680-41c3-ae09-6aaf9eced1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:16.285800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096634af-7680-41c3-ae09-6aaf9eced1d1 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:16.287485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 441/887] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:16.292490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00d153-35ec-41e3-93f4-0b2b21e2ef5e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:16.292779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00d153-35ec-41e3-93f4-0b2b21e2ef5e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:16.293058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00d153-35ec-41e3-93f4-0b2b21e2ef5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:16.311418 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ca00d153-35ec-41e3-93f4-0b2b21e2ef5e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:16.313121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ca00d153-35ec-41e3-93f4-0b2b21e2ef5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:16.313923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aaae9-3bac-40bc-a9ca-073b6dc9cf10 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:16.353665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e30374-24cd-4f84-8b8a-28225ad2fb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:16.354587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e30374-24cd-4f84-8b8a-28225ad2fb81 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:16.355047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e30374-24cd-4f84-8b8a-28225ad2fb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:16.355468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e30374-24cd-4f84-8b8a-28225ad2fb81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:16.356440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e30374-24cd-4f84-8b8a-28225ad2fb81 None admin] RBAC: Adding query filter params (interface=public, service_id=f0c6093538fd45e897ec180f08133f71, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:16.356912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e30374-24cd-4f84-8b8a-28225ad2fb81 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:16.359641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e30374-24cd-4f84-8b8a-28225ad2fb81 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:16.366581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 447/888] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jun 29 08:23:16 2022] GET /identity/v3/endpoints?service_id=f0c6093538fd45e897ec180f08133f71&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:16.371790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e30374-24cd-4f84-8b8a-28225ad2fb81 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:16.398898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 442/889] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:16.416390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00d153-35ec-41e3-93f4-0b2b21e2ef5e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:16.496261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e30374-24cd-4f84-8b8a-28225ad2fb81 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:16.511629 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:16.531934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00d153-35ec-41e3-93f4-0b2b21e2ef5e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:16.533299 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:16.541918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb00f6e-62c6-4199-aa56-d22ac160e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:16.543176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb00f6e-62c6-4199-aa56-d22ac160e98c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:16.543622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb00f6e-62c6-4199-aa56-d22ac160e98c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:16.544049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb00f6e-62c6-4199-aa56-d22ac160e98c None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:16.545203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb00f6e-62c6-4199-aa56-d22ac160e98c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:16.547040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb00f6e-62c6-4199-aa56-d22ac160e98c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:16.554825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 448/890] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jun 29 08:23:16 2022] GET /identity/v3/services => generated 1268 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:23:16.575901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38500300-559b-4ec4-bdb7-3328e30ab4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:16.576930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38500300-559b-4ec4-bdb7-3328e30ab4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:16.577306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38500300-559b-4ec4-bdb7-3328e30ab4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:16.577665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38500300-559b-4ec4-bdb7-3328e30ab4f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:16.591148 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38500300-559b-4ec4-bdb7-3328e30ab4f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-38500300-559b-4ec4-bdb7-3328e30ab4f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:16.599853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:16.600856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:16.602000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 443/891] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jun 29 08:23:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:16.607668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb00f6e-62c6-4199-aa56-d22ac160e98c None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:16.729149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb00f6e-62c6-4199-aa56-d22ac160e98c None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:16.765423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f2b51-f2f3-4341-a6e6-ef479c534dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:16.766099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f2b51-f2f3-4341-a6e6-ef479c534dfc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:16.766382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f2b51-f2f3-4341-a6e6-ef479c534dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:16.766660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f2b51-f2f3-4341-a6e6-ef479c534dfc None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:16.767646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f2b51-f2f3-4341-a6e6-ef479c534dfc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:16.767967 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f2b51-f2f3-4341-a6e6-ef479c534dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:16.769445 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f2b51-f2f3-4341-a6e6-ef479c534dfc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:16.778802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 449/892] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jun 29 08:23:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:16.784017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38500300-559b-4ec4-bdb7-3328e30ab4f1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:16.889455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38500300-559b-4ec4-bdb7-3328e30ab4f1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:16.927685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baeb940-cb32-4153-bc1d-8204b580ac22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:16.928493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeb940-cb32-4153-bc1d-8204b580ac22 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:16.928770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeb940-cb32-4153-bc1d-8204b580ac22 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:16.929038 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeb940-cb32-4153-bc1d-8204b580ac22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:16.940070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baeb940-cb32-4153-bc1d-8204b580ac22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:16.941527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baeb940-cb32-4153-bc1d-8204b580ac22 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:16.947841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-4baeb940-cb32-4153-bc1d-8204b580ac22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:16.959275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:16.960174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:16.961039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:16.961039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:16.961039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:16.961039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:16.965764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 444/893] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:16.966783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f2b51-f2f3-4341-a6e6-ef479c534dfc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:17.026394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:17.075708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f2b51-f2f3-4341-a6e6-ef479c534dfc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:17.111126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7ea0b-bf0f-4bd9-8eda-c17303a43945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:17.112307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7ea0b-bf0f-4bd9-8eda-c17303a43945 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:17.112741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7ea0b-bf0f-4bd9-8eda-c17303a43945 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:17.113158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7ea0b-bf0f-4bd9-8eda-c17303a43945 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:17.114028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7ea0b-bf0f-4bd9-8eda-c17303a43945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:17.114504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7ea0b-bf0f-4bd9-8eda-c17303a43945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:17.116048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7ea0b-bf0f-4bd9-8eda-c17303a43945 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:17.133435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 450/894] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:17.138763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baeb940-cb32-4153-bc1d-8204b580ac22 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:17.239659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baeb940-cb32-4153-bc1d-8204b580ac22 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:17.273614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866eb90-17be-4e60-ad79-bde2e3e7f498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:17.274434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866eb90-17be-4e60-ad79-bde2e3e7f498 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:17.274731 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866eb90-17be-4e60-ad79-bde2e3e7f498 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:17.275013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866eb90-17be-4e60-ad79-bde2e3e7f498 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:17.283501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866eb90-17be-4e60-ad79-bde2e3e7f498 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:17.284868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866eb90-17be-4e60-ad79-bde2e3e7f498 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a866eb90-17be-4e60-ad79-bde2e3e7f498 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:17.299178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:17.300126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:17.301054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:17.301054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:17.301054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:17.301054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:17.301054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:17.301054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:17.301054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:17.301054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:17.301054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:17.301054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:17.301054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 445/895] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jun 29 08:23:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:17.306370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7ea0b-bf0f-4bd9-8eda-c17303a43945 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:17.418624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7ea0b-bf0f-4bd9-8eda-c17303a43945 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:17.436639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:17.454667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842df224-f2ff-4ad5-8f90-862b77432657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:17.455835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842df224-f2ff-4ad5-8f90-862b77432657 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:17.456175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842df224-f2ff-4ad5-8f90-862b77432657 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:17.456465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842df224-f2ff-4ad5-8f90-862b77432657 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:17.457334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842df224-f2ff-4ad5-8f90-862b77432657 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:17.457653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842df224-f2ff-4ad5-8f90-862b77432657 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:17.459705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842df224-f2ff-4ad5-8f90-862b77432657 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:17.481222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 451/896] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jun 29 08:23:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:17.486703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866eb90-17be-4e60-ad79-bde2e3e7f498 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:17.543338 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:17.588933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866eb90-17be-4e60-ad79-bde2e3e7f498 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:17.623237 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ab7d5-ffe3-4b72-bd64-9cf9f376440c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:17.623892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ab7d5-ffe3-4b72-bd64-9cf9f376440c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:17.624167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ab7d5-ffe3-4b72-bd64-9cf9f376440c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:17.624441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ab7d5-ffe3-4b72-bd64-9cf9f376440c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:17.637999 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd17272-6e27-423a-a343-f1b155671402 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:17.638390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd17272-6e27-423a-a343-f1b155671402 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:17.638560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd17272-6e27-423a-a343-f1b155671402 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:17.639953 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 452/897] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:17.645264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508cf08-8cf7-4330-bf4b-2684bd0fe3da None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:17.645552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508cf08-8cf7-4330-bf4b-2684bd0fe3da None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:17.645838 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508cf08-8cf7-4330-bf4b-2684bd0fe3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:17.663667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c508cf08-8cf7-4330-bf4b-2684bd0fe3da None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:17.664980 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c508cf08-8cf7-4330-bf4b-2684bd0fe3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:17.694165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ab7d5-ffe3-4b72-bd64-9cf9f376440c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:17.695759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ab7d5-ffe3-4b72-bd64-9cf9f376440c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:17.752364 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 453/898] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:17.760773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:17.773579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508cf08-8cf7-4330-bf4b-2684bd0fe3da None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:17.812094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 446/899] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:17 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:23:17.819760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ab7d5-ffe3-4b72-bd64-9cf9f376440c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:17.893481 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508cf08-8cf7-4330-bf4b-2684bd0fe3da None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:17.896683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:17.932130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ab7d5-ffe3-4b72-bd64-9cf9f376440c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:17.937527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2354d0e-a777-4572-8629-23602221bb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:17.938358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2354d0e-a777-4572-8629-23602221bb5e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:17.938639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2354d0e-a777-4572-8629-23602221bb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:17.938938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2354d0e-a777-4572-8629-23602221bb5e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:17.939701 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2354d0e-a777-4572-8629-23602221bb5e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:17.941135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2354d0e-a777-4572-8629-23602221bb5e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-f2354d0e-a777-4572-8629-23602221bb5e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:17.956982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:17.957929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 08:23:17.958824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:17.967007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 454/900] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Jun 29 08:23:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:17.968096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2354d0e-a777-4572-8629-23602221bb5e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:17.975898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6025d9b-c188-4010-aa0a-ecee2c9ff1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:17.976722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6025d9b-c188-4010-aa0a-ecee2c9ff1eb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:17.977216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6025d9b-c188-4010-aa0a-ecee2c9ff1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:17.977491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6025d9b-c188-4010-aa0a-ecee2c9ff1eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:17.989165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6025d9b-c188-4010-aa0a-ecee2c9ff1eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:17.990605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6025d9b-c188-4010-aa0a-ecee2c9ff1eb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a6025d9b-c188-4010-aa0a-ecee2c9ff1eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:18.008889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:18.009772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:18.010641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:18.010641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:18.010641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:18.010641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:18.010641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 447/901] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:18.011420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d723d4-7095-4774-8de1-f55fd01f7790 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.011591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d723d4-7095-4774-8de1-f55fd01f7790 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.011894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d723d4-7095-4774-8de1-f55fd01f7790 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.013178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 448/902] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:18.015493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d723d4-7095-4774-8de1-f55fd01f7790 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:18.025635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:18.079401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2354d0e-a777-4572-8629-23602221bb5e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:18.117376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa3d17-9d2f-4847-a431-29fe2e6b6ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:18.118188 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3d17-9d2f-4847-a431-29fe2e6b6ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.118804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3d17-9d2f-4847-a431-29fe2e6b6ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.119089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3d17-9d2f-4847-a431-29fe2e6b6ba5 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.120077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa3d17-9d2f-4847-a431-29fe2e6b6ba5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:18.120392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa3d17-9d2f-4847-a431-29fe2e6b6ba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:18.122469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa3d17-9d2f-4847-a431-29fe2e6b6ba5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:18.124962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d723d4-7095-4774-8de1-f55fd01f7790 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:18.132764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 455/903] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jun 29 08:23:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:18.132764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb708d5-1428-4a5a-b8d8-0a0da6185d73 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.132764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb708d5-1428-4a5a-b8d8-0a0da6185d73 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.132764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb708d5-1428-4a5a-b8d8-0a0da6185d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.150793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0cb708d5-1428-4a5a-b8d8-0a0da6185d73 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:18.151992 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0cb708d5-1428-4a5a-b8d8-0a0da6185d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:18.160142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df8946-8e68-4f07-b314-25aea39940f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:18.160825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df8946-8e68-4f07-b314-25aea39940f2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.161113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df8946-8e68-4f07-b314-25aea39940f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.161393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df8946-8e68-4f07-b314-25aea39940f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.162132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df8946-8e68-4f07-b314-25aea39940f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:18.162446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df8946-8e68-4f07-b314-25aea39940f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:18.163918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df8946-8e68-4f07-b314-25aea39940f2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:18.176177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 449/904] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:18.181605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df8946-8e68-4f07-b314-25aea39940f2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:18.182020 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:18.226893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 456/905] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:18.227541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb708d5-1428-4a5a-b8d8-0a0da6185d73 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:18.263986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:18.301927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df8946-8e68-4f07-b314-25aea39940f2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:18.334681 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb708d5-1428-4a5a-b8d8-0a0da6185d73 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:18.336738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85f3cc-4387-4966-8c71-9f7950c2644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:18.337407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85f3cc-4387-4966-8c71-9f7950c2644c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.337692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85f3cc-4387-4966-8c71-9f7950c2644c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.337991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85f3cc-4387-4966-8c71-9f7950c2644c None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.339090 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85f3cc-4387-4966-8c71-9f7950c2644c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:18.339400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85f3cc-4387-4966-8c71-9f7950c2644c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:18.340837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85f3cc-4387-4966-8c71-9f7950c2644c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:18.346356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 450/906] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jun 29 08:23:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:18.347496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85f3cc-4387-4966-8c71-9f7950c2644c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:18.374531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2b7ab-7634-4510-a534-6397357e91d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:18.375387 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2b7ab-7634-4510-a534-6397357e91d3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.375674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2b7ab-7634-4510-a534-6397357e91d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.375958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2b7ab-7634-4510-a534-6397357e91d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.382932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2b7ab-7634-4510-a534-6397357e91d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:18.384393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2b7ab-7634-4510-a534-6397357e91d3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6ec2b7ab-7634-4510-a534-6397357e91d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:18.398596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:18.399722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:18.400605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:18.400605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:18.400605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:18.400605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:18.400605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:18.400605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:18.400605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:18.400605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:18.400605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 457/907] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:18.400605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2b7ab-7634-4510-a534-6397357e91d3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:18.428102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:18.471019 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85f3cc-4387-4966-8c71-9f7950c2644c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:18.500627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:18.508331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12845615-2e22-4b1d-b664-fb858a1b14c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:18.509016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12845615-2e22-4b1d-b664-fb858a1b14c2 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.509286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12845615-2e22-4b1d-b664-fb858a1b14c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.509564 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12845615-2e22-4b1d-b664-fb858a1b14c2 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.510294 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12845615-2e22-4b1d-b664-fb858a1b14c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:18.511712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12845615-2e22-4b1d-b664-fb858a1b14c2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:18.515439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2b7ab-7634-4510-a534-6397357e91d3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:18.538720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 451/908] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:23:18 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:18.538720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12845615-2e22-4b1d-b664-fb858a1b14c2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:18.577328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd2858-e3a3-442a-a971-0faa1cee3752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:18.578172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd2858-e3a3-442a-a971-0faa1cee3752 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.578456 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd2858-e3a3-442a-a971-0faa1cee3752 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.579165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd2858-e3a3-442a-a971-0faa1cee3752 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.580465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd2858-e3a3-442a-a971-0faa1cee3752 None admin] RBAC: Adding query filter params (interface=public, service_id=754d00a9dd7f4970b7c3e69870be40cf, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:18.580786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd2858-e3a3-442a-a971-0faa1cee3752 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:18.582815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd2858-e3a3-442a-a971-0faa1cee3752 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:18.591071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 458/909] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jun 29 08:23:18 2022] GET /identity/v3/endpoints?service_id=754d00a9dd7f4970b7c3e69870be40cf&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:18.596991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd2858-e3a3-442a-a971-0faa1cee3752 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:18.665605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12845615-2e22-4b1d-b664-fb858a1b14c2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:18.706257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:18.709264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42d7b7-a381-4437-a675-5dc12ab8bbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:18.709264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d7b7-a381-4437-a675-5dc12ab8bbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.709264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d7b7-a381-4437-a675-5dc12ab8bbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.709563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d7b7-a381-4437-a675-5dc12ab8bbf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.710288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42d7b7-a381-4437-a675-5dc12ab8bbf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:18.710605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42d7b7-a381-4437-a675-5dc12ab8bbf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:18.712792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42d7b7-a381-4437-a675-5dc12ab8bbf7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:18.729085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd2858-e3a3-442a-a971-0faa1cee3752 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:18.730187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 452/910] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:18.731579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c995ca-21fb-408e-8af5-9f424bce662b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.731870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c995ca-21fb-408e-8af5-9f424bce662b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.732146 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c995ca-21fb-408e-8af5-9f424bce662b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.733491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 453/911] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:18.739904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c995ca-21fb-408e-8af5-9f424bce662b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:18.771095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4848db8-00e1-46e1-b46b-7bb6783ed475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:18.771931 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4848db8-00e1-46e1-b46b-7bb6783ed475 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.772492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4848db8-00e1-46e1-b46b-7bb6783ed475 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.773018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4848db8-00e1-46e1-b46b-7bb6783ed475 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.773868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4848db8-00e1-46e1-b46b-7bb6783ed475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:18.775292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4848db8-00e1-46e1-b46b-7bb6783ed475 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:18.787343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 459/912] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jun 29 08:23:18 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:23:18.787343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cf032-6243-4dd2-9541-99df4fa76f00 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.787343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cf032-6243-4dd2-9541-99df4fa76f00 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.787343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cf032-6243-4dd2-9541-99df4fa76f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.793486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:18.812699 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-544cf032-6243-4dd2-9541-99df4fa76f00 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:18.814299 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-544cf032-6243-4dd2-9541-99df4fa76f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:18.858108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c995ca-21fb-408e-8af5-9f424bce662b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:18.897335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 460/913] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:18.899073 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df210d1-9c0b-4884-bdd7-f53ff1c94a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:18.899775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df210d1-9c0b-4884-bdd7-f53ff1c94a5a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:18.900058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df210d1-9c0b-4884-bdd7-f53ff1c94a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:18.900335 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df210d1-9c0b-4884-bdd7-f53ff1c94a5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:18.901186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df210d1-9c0b-4884-bdd7-f53ff1c94a5a None admin] RBAC: Adding query filter params (role.id=f21b1b6d7f944363b8107baa2cc893ca, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=f320b56bf7ff4c36bbd9730e688612d5) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:18.901504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df210d1-9c0b-4884-bdd7-f53ff1c94a5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:18.903082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df210d1-9c0b-4884-bdd7-f53ff1c94a5a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:18.910230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 454/914] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:18 2022] GET /identity/v3/role_assignments?user.id=f320b56bf7ff4c36bbd9730e688612d5&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=f21b1b6d7f944363b8107baa2cc893ca => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:18.921197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cf032-6243-4dd2-9541-99df4fa76f00 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:19.018355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:19.044168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cf032-6243-4dd2-9541-99df4fa76f00 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:19.082912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b03bb-b8d8-4f96-b121-612f701eb8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:19.083708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b03bb-b8d8-4f96-b121-612f701eb8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:19.083995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b03bb-b8d8-4f96-b121-612f701eb8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:19.084271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b03bb-b8d8-4f96-b121-612f701eb8a0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:19.084942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b03bb-b8d8-4f96-b121-612f701eb8a0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:19.086330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b03bb-b8d8-4f96-b121-612f701eb8a0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ac2b03bb-b8d8-4f96-b121-612f701eb8a0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:19.099378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:19.100392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:19.101470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 461/915] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:23:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:19.106585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df210d1-9c0b-4884-bdd7-f53ff1c94a5a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:19.216079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df210d1-9c0b-4884-bdd7-f53ff1c94a5a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:19.231774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:19.253520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed157781-4308-4487-99f5-31ef966d285c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:19.254450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed157781-4308-4487-99f5-31ef966d285c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:19.254941 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed157781-4308-4487-99f5-31ef966d285c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:19.255362 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed157781-4308-4487-99f5-31ef966d285c None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:19.256224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed157781-4308-4487-99f5-31ef966d285c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:19.256677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed157781-4308-4487-99f5-31ef966d285c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:19.258195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed157781-4308-4487-99f5-31ef966d285c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:19.263908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 455/916] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:23:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:19.272470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b03bb-b8d8-4f96-b121-612f701eb8a0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:19.376296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b03bb-b8d8-4f96-b121-612f701eb8a0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:19.410748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccda18b-f51b-4f06-9ed1-735c01ec7943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:19.411423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda18b-f51b-4f06-9ed1-735c01ec7943 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:19.411705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda18b-f51b-4f06-9ed1-735c01ec7943 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:19.411978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccda18b-f51b-4f06-9ed1-735c01ec7943 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:19.412693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccda18b-f51b-4f06-9ed1-735c01ec7943 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:19.413003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccda18b-f51b-4f06-9ed1-735c01ec7943 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:19.414382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccda18b-f51b-4f06-9ed1-735c01ec7943 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:19.420678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 462/917] 10.222.0.27 () {56 vars in 1114 bytes} [Wed Jun 29 08:23:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:19.429791 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed157781-4308-4487-99f5-31ef966d285c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:19.540807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed157781-4308-4487-99f5-31ef966d285c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:19.574899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167bd1e-7b54-4191-9d58-d93f13eafa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:19.575620 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167bd1e-7b54-4191-9d58-d93f13eafa3c None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:19.575904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167bd1e-7b54-4191-9d58-d93f13eafa3c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:19.576309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167bd1e-7b54-4191-9d58-d93f13eafa3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:19.577104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167bd1e-7b54-4191-9d58-d93f13eafa3c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:19.578485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167bd1e-7b54-4191-9d58-d93f13eafa3c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:19.601298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 456/918] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jun 29 08:23:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:20.653160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807091e-7e5c-4feb-b0b5-1be1c1294eb0 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:20.653691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807091e-7e5c-4feb-b0b5-1be1c1294eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:20.653691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807091e-7e5c-4feb-b0b5-1be1c1294eb0 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:20.655559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 463/919] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:20.661091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94563f61-0552-407a-842f-625e1c925b7b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:20.661513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94563f61-0552-407a-842f-625e1c925b7b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:20.661683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94563f61-0552-407a-842f-625e1c925b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:20.679107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-94563f61-0552-407a-842f-625e1c925b7b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:20.680303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-94563f61-0552-407a-842f-625e1c925b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:20.708085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:20.757677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 457/920] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:23:20.781222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807091e-7e5c-4feb-b0b5-1be1c1294eb0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:20.785374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:20.891391 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807091e-7e5c-4feb-b0b5-1be1c1294eb0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:20.920508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7a88d-0960-4ea2-a7ca-a83a71bec45f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:20.920892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7a88d-0960-4ea2-a7ca-a83a71bec45f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:20.921060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7a88d-0960-4ea2-a7ca-a83a71bec45f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:20.922652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 458/921] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:20.927872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00457144-5bee-429e-a30a-c418e8462a23 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:20.928158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00457144-5bee-429e-a30a-c418e8462a23 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:20.928435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00457144-5bee-429e-a30a-c418e8462a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:20.929069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343905f2-6b00-428a-8902-c97ce6242858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:20.930091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343905f2-6b00-428a-8902-c97ce6242858 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:20.930524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343905f2-6b00-428a-8902-c97ce6242858 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:20.930970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343905f2-6b00-428a-8902-c97ce6242858 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:20.931795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343905f2-6b00-428a-8902-c97ce6242858 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:20.933309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343905f2-6b00-428a-8902-c97ce6242858 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:20.945614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-00457144-5bee-429e-a30a-c418e8462a23 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-343905f2-6b00-428a-8902-c97ce6242858 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:20.946092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:20.947289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:20.948312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:20.949567 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-00457144-5bee-429e-a30a-c418e8462a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:20.950977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 464/922] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jun 29 08:23:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:20.962624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343905f2-6b00-428a-8902-c97ce6242858 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:21.019829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 459/923] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:21.021162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57503dc0-52fd-4ed5-a51b-10e65879693b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.021458 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57503dc0-52fd-4ed5-a51b-10e65879693b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.021740 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57503dc0-52fd-4ed5-a51b-10e65879693b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.027204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 460/924] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:21.028448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f199b-ca96-42e2-90a5-01fb87c85f3b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.028734 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f199b-ca96-42e2-90a5-01fb87c85f3b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.029014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f199b-ca96-42e2-90a5-01fb87c85f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.041759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:21.049301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5a1f199b-ca96-42e2-90a5-01fb87c85f3b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:21.050494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5a1f199b-ca96-42e2-90a5-01fb87c85f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:21.080262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343905f2-6b00-428a-8902-c97ce6242858 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:21.113381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397c24a-d42f-4655-a571-59e6e3233055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:21.114050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397c24a-d42f-4655-a571-59e6e3233055 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.114324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397c24a-d42f-4655-a571-59e6e3233055 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.114598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397c24a-d42f-4655-a571-59e6e3233055 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.115348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397c24a-d42f-4655-a571-59e6e3233055 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:21.115648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397c24a-d42f-4655-a571-59e6e3233055 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:21.117014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397c24a-d42f-4655-a571-59e6e3233055 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:21.121732 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 465/925] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jun 29 08:23:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:21.130725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397c24a-d42f-4655-a571-59e6e3233055 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:21.130909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 461/926] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:21.130909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f199b-ca96-42e2-90a5-01fb87c85f3b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:21.157477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:21.227476 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397c24a-d42f-4655-a571-59e6e3233055 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:21.243840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f199b-ca96-42e2-90a5-01fb87c85f3b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:21.258634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:21.265048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94f49c-8519-48b6-a838-f2afc05ce5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:21.265986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94f49c-8519-48b6-a838-f2afc05ce5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.266268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94f49c-8519-48b6-a838-f2afc05ce5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.266638 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94f49c-8519-48b6-a838-f2afc05ce5e9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.277270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec5918-7095-447a-bfb9-2f8d67b13c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:21.277619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94f49c-8519-48b6-a838-f2afc05ce5e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:21.277866 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec5918-7095-447a-bfb9-2f8d67b13c14 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.278146 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec5918-7095-447a-bfb9-2f8d67b13c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.278418 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec5918-7095-447a-bfb9-2f8d67b13c14 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.279161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec5918-7095-447a-bfb9-2f8d67b13c14 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:21.279468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec5918-7095-447a-bfb9-2f8d67b13c14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:21.280834 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec5918-7095-447a-bfb9-2f8d67b13c14 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:21.290611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 462/927] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jun 29 08:23:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4e94f49c-8519-48b6-a838-f2afc05ce5e9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:21.290805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:21.291671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:21.292499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:21.293149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec5918-7095-447a-bfb9-2f8d67b13c14 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:21.293227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 466/928] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jun 29 08:23:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:21.293227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94f49c-8519-48b6-a838-f2afc05ce5e9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:21.378389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:21.394811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec5918-7095-447a-bfb9-2f8d67b13c14 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:21.400691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94f49c-8519-48b6-a838-f2afc05ce5e9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:21.429924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382d9e2-9b8a-43a5-8c54-47a15a7983a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:21.430761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382d9e2-9b8a-43a5-8c54-47a15a7983a0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.431037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382d9e2-9b8a-43a5-8c54-47a15a7983a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.431314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382d9e2-9b8a-43a5-8c54-47a15a7983a0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.431974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382d9e2-9b8a-43a5-8c54-47a15a7983a0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:21.433327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382d9e2-9b8a-43a5-8c54-47a15a7983a0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:21.437383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926f1ee-7db1-41f3-a2db-f3479914285a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:21.438238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926f1ee-7db1-41f3-a2db-f3479914285a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.438716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926f1ee-7db1-41f3-a2db-f3479914285a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.439354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926f1ee-7db1-41f3-a2db-f3479914285a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.440421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926f1ee-7db1-41f3-a2db-f3479914285a None admin] RBAC: Adding query filter params (interface=public, service_id=fc733a28078d45aa9ce69822860b0e32, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:21.440968 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926f1ee-7db1-41f3-a2db-f3479914285a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:21.442512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926f1ee-7db1-41f3-a2db-f3479914285a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-7382d9e2-9b8a-43a5-8c54-47a15a7983a0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:21.445481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:21.446391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:21.447257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 463/929] 10.222.0.27 () {56 vars in 1132 bytes} [Wed Jun 29 08:23:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:21.448538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382d9e2-9b8a-43a5-8c54-47a15a7983a0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:21.459060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 467/930] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jun 29 08:23:21 2022] GET /identity/v3/endpoints?service_id=fc733a28078d45aa9ce69822860b0e32&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:21.459060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926f1ee-7db1-41f3-a2db-f3479914285a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:21.517360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:21.569342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926f1ee-7db1-41f3-a2db-f3479914285a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:21.570428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382d9e2-9b8a-43a5-8c54-47a15a7983a0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:21.605774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8a384-7eb1-4856-8cd0-023d4fb4871c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:21.606443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8a384-7eb1-4856-8cd0-023d4fb4871c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.606734 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8a384-7eb1-4856-8cd0-023d4fb4871c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.607028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8a384-7eb1-4856-8cd0-023d4fb4871c None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.607777 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8a384-7eb1-4856-8cd0-023d4fb4871c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:21.607881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81824f-2050-4ddc-8505-74b630f29b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:21.608061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8a384-7eb1-4856-8cd0-023d4fb4871c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:21.608549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81824f-2050-4ddc-8505-74b630f29b88 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.608842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81824f-2050-4ddc-8505-74b630f29b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.609122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81824f-2050-4ddc-8505-74b630f29b88 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.609435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8a384-7eb1-4856-8cd0-023d4fb4871c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:21.609839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81824f-2050-4ddc-8505-74b630f29b88 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:21.610152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81824f-2050-4ddc-8505-74b630f29b88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:21.612047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b81824f-2050-4ddc-8505-74b630f29b88 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:21.618179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 468/931] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jun 29 08:23:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:21.618639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 464/932] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jun 29 08:23:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:21.618975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81824f-2050-4ddc-8505-74b630f29b88 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:21.623695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8a384-7eb1-4856-8cd0-023d4fb4871c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:21.637208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:21.731137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8a384-7eb1-4856-8cd0-023d4fb4871c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:21.734334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81824f-2050-4ddc-8505-74b630f29b88 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:21.753743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:21.765418 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851cd80-f3c1-412e-a356-af01ededf919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:21.766219 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cd80-f3c1-412e-a356-af01ededf919 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.766492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cd80-f3c1-412e-a356-af01ededf919 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.766796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cd80-f3c1-412e-a356-af01ededf919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.769123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73825953-0d6d-45e5-b49d-bd7bf7a0a49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:21.769787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73825953-0d6d-45e5-b49d-bd7bf7a0a49a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.770069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73825953-0d6d-45e5-b49d-bd7bf7a0a49a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.770347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73825953-0d6d-45e5-b49d-bd7bf7a0a49a None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.771090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73825953-0d6d-45e5-b49d-bd7bf7a0a49a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:21.772473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73825953-0d6d-45e5-b49d-bd7bf7a0a49a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:21.777597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851cd80-f3c1-412e-a356-af01ededf919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:21.778712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 469/933] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jun 29 08:23:21 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:23:21.779077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851cd80-f3c1-412e-a356-af01ededf919 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:21.779503 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73825953-0d6d-45e5-b49d-bd7bf7a0a49a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3851cd80-f3c1-412e-a356-af01ededf919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:21.806750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:21.808003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:21.809108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:21.809108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:21.809108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:21.809108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:21.809108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 465/934] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:21.809475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f951b4f-8f79-4001-bcf1-7a9f1960e509 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.809731 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f951b4f-8f79-4001-bcf1-7a9f1960e509 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.810013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f951b4f-8f79-4001-bcf1-7a9f1960e509 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.814898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 466/935] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:21.814898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f951b4f-8f79-4001-bcf1-7a9f1960e509 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:21.884572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:21.903567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73825953-0d6d-45e5-b49d-bd7bf7a0a49a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:21.931135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f951b4f-8f79-4001-bcf1-7a9f1960e509 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:21.940025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e5aa8-53aa-4329-9756-92c433cfdd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:21.940700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e5aa8-53aa-4329-9756-92c433cfdd75 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.941547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e5aa8-53aa-4329-9756-92c433cfdd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.941547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e5aa8-53aa-4329-9756-92c433cfdd75 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.941955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e5aa8-53aa-4329-9756-92c433cfdd75 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:21.942264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e5aa8-53aa-4329-9756-92c433cfdd75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:21.943663 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e5aa8-53aa-4329-9756-92c433cfdd75 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:21.954759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 470/936] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jun 29 08:23:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:21.954759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348edfd2-c37d-4825-a836-7d18e17bc28d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.954759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348edfd2-c37d-4825-a836-7d18e17bc28d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.954759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348edfd2-c37d-4825-a836-7d18e17bc28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.968951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-348edfd2-c37d-4825-a836-7d18e17bc28d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:21.970133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-348edfd2-c37d-4825-a836-7d18e17bc28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:21.976867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399fa96-84b5-492e-a6cd-c5a1563c04a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:21.977556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399fa96-84b5-492e-a6cd-c5a1563c04a6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:21.977846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399fa96-84b5-492e-a6cd-c5a1563c04a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:21.978540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399fa96-84b5-492e-a6cd-c5a1563c04a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:21.979324 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399fa96-84b5-492e-a6cd-c5a1563c04a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:21.979638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399fa96-84b5-492e-a6cd-c5a1563c04a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:21.981710 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399fa96-84b5-492e-a6cd-c5a1563c04a6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:21.994409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 467/937] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:21.995085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399fa96-84b5-492e-a6cd-c5a1563c04a6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:22.025364 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:22.052460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 471/938] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:22.053387 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348edfd2-c37d-4825-a836-7d18e17bc28d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:22.112449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399fa96-84b5-492e-a6cd-c5a1563c04a6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:22.138423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:22.155379 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b22ad-c460-41c6-ab9f-9dc605d756d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:22.156041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b22ad-c460-41c6-ab9f-9dc605d756d9 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:22.156321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b22ad-c460-41c6-ab9f-9dc605d756d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:22.156622 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b22ad-c460-41c6-ab9f-9dc605d756d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:22.157324 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b22ad-c460-41c6-ab9f-9dc605d756d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:22.158863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b22ad-c460-41c6-ab9f-9dc605d756d9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:22.163796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348edfd2-c37d-4825-a836-7d18e17bc28d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:22.186738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 468/939] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jun 29 08:23:21 2022] POST /identity/v3/endpoints => generated 347 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:22.186738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b22ad-c460-41c6-ab9f-9dc605d756d9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:22.219450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255af70a-92f7-4ff2-800c-f7f738a2cb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:22.220683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255af70a-92f7-4ff2-800c-f7f738a2cb1d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:22.221072 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255af70a-92f7-4ff2-800c-f7f738a2cb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:22.221520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255af70a-92f7-4ff2-800c-f7f738a2cb1d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:22.231621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255af70a-92f7-4ff2-800c-f7f738a2cb1d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:22.233360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255af70a-92f7-4ff2-800c-f7f738a2cb1d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-255af70a-92f7-4ff2-800c-f7f738a2cb1d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:22.252143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:22.253224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:22.254161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:22.254161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:22.254161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:22.254161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:22.254161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:22.254161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:22.254161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:22.254161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:22.254161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:22.254161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:22.254161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 472/940] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jun 29 08:23:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:22.260511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255af70a-92f7-4ff2-800c-f7f738a2cb1d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:22.291038 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b22ad-c460-41c6-ab9f-9dc605d756d9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:22.319706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:22.328706 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa9fac-a7b0-427a-b71e-257964a8cfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:22.329394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa9fac-a7b0-427a-b71e-257964a8cfc6 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:22.329712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa9fac-a7b0-427a-b71e-257964a8cfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:22.330166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa9fac-a7b0-427a-b71e-257964a8cfc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:22.331093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa9fac-a7b0-427a-b71e-257964a8cfc6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:22.332566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa9fac-a7b0-427a-b71e-257964a8cfc6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:22.333976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: WARNING keystone.server.flask.common [None req-eaaa9fac-a7b0-427a-b71e-257964a8cfc6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 29 08:23:22.369188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.sql.core [None req-eaaa9fac-a7b0-427a-b71e-257964a8cfc6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 08:23:22.369188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 29 08:23:22.369188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [parameters: {'id': '13de32683fef4038920fa8fa9da1e592', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 29 08:23:22.369188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80130) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-eaaa9fac-a7b0-427a-b71e-257964a8cfc6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 13de32683fef4038920fa8fa9da1e592.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 13de32683fef4038920fa8fa9da1e592. Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application conn.query(q) Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 08:23:22.377905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application result.read() Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 29 08:23:22.378767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application next(self.gen) Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application yield resource Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application next(self.gen) Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise self.value Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application session.commit() Jun 29 08:23:22.379796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self.session.flush() Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application compat.raise_( Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 08:23:22.380724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 08:23:22.381610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application conn.query(q) Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application result.read() Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 08:23:22.382402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [parameters: {'id': '13de32683fef4038920fa8fa9da1e592', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:22.385288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:22.387221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:22.391218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:22.391218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 29 08:23:22.391218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 29 08:23:22.391218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:22.391218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:22.391218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 29 08:23:22.391218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 29 08:23:22.391218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 13de32683fef4038920fa8fa9da1e592. Jun 29 08:23:22.391218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:22.391218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 469/941] 10.222.0.27 () {60 vars in 1140 bytes} [Wed Jun 29 08:23:22 2022] POST /identity/v3/projects => generated 262 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:22.395417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa9fac-a7b0-427a-b71e-257964a8cfc6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:22.406301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255af70a-92f7-4ff2-800c-f7f738a2cb1d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:22.443649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:22.461979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91abc09-14ab-464e-b51b-6231e4e23e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:22.463023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91abc09-14ab-464e-b51b-6231e4e23e9e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:22.463871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91abc09-14ab-464e-b51b-6231e4e23e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:22.464332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91abc09-14ab-464e-b51b-6231e4e23e9e None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:22.465462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91abc09-14ab-464e-b51b-6231e4e23e9e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:22.466027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91abc09-14ab-464e-b51b-6231e4e23e9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:22.467747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91abc09-14ab-464e-b51b-6231e4e23e9e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:22.487673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 473/942] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jun 29 08:23:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:22.493321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91abc09-14ab-464e-b51b-6231e4e23e9e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:22.524890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa9fac-a7b0-427a-b71e-257964a8cfc6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:22.561609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d88bb0-a96f-4dd0-8318-5187da6615fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:22.562417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d88bb0-a96f-4dd0-8318-5187da6615fb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:22.562718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d88bb0-a96f-4dd0-8318-5187da6615fb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:22.563375 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d88bb0-a96f-4dd0-8318-5187da6615fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:22.570309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d88bb0-a96f-4dd0-8318-5187da6615fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:22.572106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d88bb0-a96f-4dd0-8318-5187da6615fb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-c2d88bb0-a96f-4dd0-8318-5187da6615fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:22.586504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:22.587530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:22.588789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:22.588789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:22.588789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:22.588789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:22.588789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:22.588789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:22.588789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:22.588789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:22.588789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 470/943] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:22.593668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d88bb0-a96f-4dd0-8318-5187da6615fb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:22.611705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:22.624345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91abc09-14ab-464e-b51b-6231e4e23e9e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:22.671302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271e856-f8c6-404b-b67c-6d7198273962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:22.671640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271e856-f8c6-404b-b67c-6d7198273962 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:22.671913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271e856-f8c6-404b-b67c-6d7198273962 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:22.672193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271e856-f8c6-404b-b67c-6d7198273962 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:22.687299 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271e856-f8c6-404b-b67c-6d7198273962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:22.690038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271e856-f8c6-404b-b67c-6d7198273962 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:22.698301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:22.706003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d88bb0-a96f-4dd0-8318-5187da6615fb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4271e856-f8c6-404b-b67c-6d7198273962 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:22.715368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:22.716467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:22.717544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:22.717544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:22.717544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:22.717544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:22.724248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 474/944] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:22.724947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271e856-f8c6-404b-b67c-6d7198273962 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:22.756788 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7373231-4982-49b4-ab71-b04bf23a4748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:22.758751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7373231-4982-49b4-ab71-b04bf23a4748 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:22.759195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7373231-4982-49b4-ab71-b04bf23a4748 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:22.759669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7373231-4982-49b4-ab71-b04bf23a4748 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:22.761028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7373231-4982-49b4-ab71-b04bf23a4748 None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:22.761675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7373231-4982-49b4-ab71-b04bf23a4748 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:22.763385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7373231-4982-49b4-ab71-b04bf23a4748 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:22.789377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 471/945] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:23:22 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:22.837987 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271e856-f8c6-404b-b67c-6d7198273962 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:22.877049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04f799-9b8b-4c3b-998f-183087319b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:22.878022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04f799-9b8b-4c3b-998f-183087319b57 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:22.878510 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04f799-9b8b-4c3b-998f-183087319b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:22.878963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04f799-9b8b-4c3b-998f-183087319b57 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:22.879875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04f799-9b8b-4c3b-998f-183087319b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:22.880331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04f799-9b8b-4c3b-998f-183087319b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:22.881934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04f799-9b8b-4c3b-998f-183087319b57 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:22.895653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 475/946] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:22.901958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7373231-4982-49b4-ab71-b04bf23a4748 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:23.011495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7373231-4982-49b4-ab71-b04bf23a4748 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:23.047305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91c1ef-6712-4615-91c0-d6438910d05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:23.048356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91c1ef-6712-4615-91c0-d6438910d05e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:23.048823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91c1ef-6712-4615-91c0-d6438910d05e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:23.049246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91c1ef-6712-4615-91c0-d6438910d05e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:23.056508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91c1ef-6712-4615-91c0-d6438910d05e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:23.058108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91c1ef-6712-4615-91c0-d6438910d05e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-dc91c1ef-6712-4615-91c0-d6438910d05e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:23.072433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:23.073533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:23.074608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:23.074608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:23.074608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:23.074608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:23.074608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:23.074608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:23.074608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:23.074608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:23.076672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 472/947] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:23.082519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04f799-9b8b-4c3b-998f-183087319b57 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:23.090739 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:23.191902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04f799-9b8b-4c3b-998f-183087319b57 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:23.229879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319ee4b-4ab5-46b1-b8eb-b4202ac3b849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:23.230910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319ee4b-4ab5-46b1-b8eb-b4202ac3b849 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:23.231385 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319ee4b-4ab5-46b1-b8eb-b4202ac3b849 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:23.231901 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319ee4b-4ab5-46b1-b8eb-b4202ac3b849 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:23.232885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319ee4b-4ab5-46b1-b8eb-b4202ac3b849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:23.233392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319ee4b-4ab5-46b1-b8eb-b4202ac3b849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:23.235057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319ee4b-4ab5-46b1-b8eb-b4202ac3b849 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:23.248189 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 476/948] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:23.254386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91c1ef-6712-4615-91c0-d6438910d05e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:23.271814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:23.359169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91c1ef-6712-4615-91c0-d6438910d05e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:23.394295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a0bbc-4441-4413-aa75-d2360fd3426b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:23.395642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a0bbc-4441-4413-aa75-d2360fd3426b None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:23.396143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a0bbc-4441-4413-aa75-d2360fd3426b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:23.396618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a0bbc-4441-4413-aa75-d2360fd3426b None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/f320b56bf7ff4c36bbd9730e688612d5/roles/f21b1b6d7f944363b8107baa2cc893ca` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:23.399264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a0bbc-4441-4413-aa75-d2360fd3426b None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=f320b56bf7ff4c36bbd9730e688612d5, role_id=f21b1b6d7f944363b8107baa2cc893ca)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:23.401068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a0bbc-4441-4413-aa75-d2360fd3426b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:23.421758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 473/949] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:23:23 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/f320b56bf7ff4c36bbd9730e688612d5/roles/f21b1b6d7f944363b8107baa2cc893ca => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:23:23.839939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cb313-331d-4d6d-89c9-bcfcd4b3f803 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:23.840507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cb313-331d-4d6d-89c9-bcfcd4b3f803 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:23.840507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820cb313-331d-4d6d-89c9-bcfcd4b3f803 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:23.842118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 477/950] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:23.847984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a39e-14be-484d-ada9-85678b5926b3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:23.848407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a39e-14be-484d-ada9-85678b5926b3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:23.848592 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7a39e-14be-484d-ada9-85678b5926b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:23.865880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c1b7a39e-14be-484d-ada9-85678b5926b3 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:23.867426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c1b7a39e-14be-484d-ada9-85678b5926b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:23.942819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 474/951] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:23.960827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cb313-331d-4d6d-89c9-bcfcd4b3f803 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:24.024558 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:24.063939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820cb313-331d-4d6d-89c9-bcfcd4b3f803 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:24.099370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a3294-d3db-4bc2-943c-c2d9de620092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:24.100551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a3294-d3db-4bc2-943c-c2d9de620092 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:24.101045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a3294-d3db-4bc2-943c-c2d9de620092 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:24.101509 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a3294-d3db-4bc2-943c-c2d9de620092 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:24.102389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a3294-d3db-4bc2-943c-c2d9de620092 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:24.104083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a3294-d3db-4bc2-943c-c2d9de620092 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-413a3294-d3db-4bc2-943c-c2d9de620092 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:24.117513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:24.118618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:24.119498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:24.122440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 478/952] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jun 29 08:23:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:24.128661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7a39e-14be-484d-ada9-85678b5926b3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:24.132133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:24.230859 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7a39e-14be-484d-ada9-85678b5926b3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:24.265658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d75c47-fd3f-463c-9bfb-efcde39fbaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:24.266330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d75c47-fd3f-463c-9bfb-efcde39fbaaf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:24.266613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d75c47-fd3f-463c-9bfb-efcde39fbaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:24.266912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d75c47-fd3f-463c-9bfb-efcde39fbaaf None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:24.267680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d75c47-fd3f-463c-9bfb-efcde39fbaaf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:24.267987 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d75c47-fd3f-463c-9bfb-efcde39fbaaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:24.269351 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d75c47-fd3f-463c-9bfb-efcde39fbaaf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:24.278805 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 475/953] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jun 29 08:23:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:24.280362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a3294-d3db-4bc2-943c-c2d9de620092 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:24.351099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac70f32-507c-4e6d-a0dd-695a49827fd0 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:24.351277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac70f32-507c-4e6d-a0dd-695a49827fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:24.351554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac70f32-507c-4e6d-a0dd-695a49827fd0 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:24.353193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 476/954] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:24.357824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d6428-f2b1-4eff-a791-dac3318b1632 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:24.358109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d6428-f2b1-4eff-a791-dac3318b1632 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:24.358387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d6428-f2b1-4eff-a791-dac3318b1632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:24.378309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-884d6428-f2b1-4eff-a791-dac3318b1632 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:24.380152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-884d6428-f2b1-4eff-a791-dac3318b1632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:24.382646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a3294-d3db-4bc2-943c-c2d9de620092 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:24.415083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:24.425444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b5aa1-8153-4ec5-a2f8-c95b161626d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:24.426116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b5aa1-8153-4ec5-a2f8-c95b161626d4 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:24.426398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b5aa1-8153-4ec5-a2f8-c95b161626d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:24.426662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b5aa1-8153-4ec5-a2f8-c95b161626d4 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:24.427402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b5aa1-8153-4ec5-a2f8-c95b161626d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:24.427709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b5aa1-8153-4ec5-a2f8-c95b161626d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:24.429531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b5aa1-8153-4ec5-a2f8-c95b161626d4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:24.436893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 479/955] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jun 29 08:23:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:24.442676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b5aa1-8153-4ec5-a2f8-c95b161626d4 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:24.470106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:24.477525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 477/956] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:23:24.499185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d6428-f2b1-4eff-a791-dac3318b1632 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:24.561476 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b5aa1-8153-4ec5-a2f8-c95b161626d4 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:24.607516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631f086-e7a4-4352-9f25-2bcd5820e691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:24.609024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631f086-e7a4-4352-9f25-2bcd5820e691 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:24.609454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631f086-e7a4-4352-9f25-2bcd5820e691 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:24.609867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631f086-e7a4-4352-9f25-2bcd5820e691 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:24.611003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631f086-e7a4-4352-9f25-2bcd5820e691 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:24.613316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b631f086-e7a4-4352-9f25-2bcd5820e691 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:24.617374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d6428-f2b1-4eff-a791-dac3318b1632 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:24.654727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 480/957] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jun 29 08:23:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:23:24.670761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342b226-f6fa-4a57-8e54-9a82c15e8a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:24.671583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b226-f6fa-4a57-8e54-9a82c15e8a2a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:24.671868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b226-f6fa-4a57-8e54-9a82c15e8a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:24.672140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342b226-f6fa-4a57-8e54-9a82c15e8a2a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:24.672803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342b226-f6fa-4a57-8e54-9a82c15e8a2a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:24.674601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342b226-f6fa-4a57-8e54-9a82c15e8a2a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6342b226-f6fa-4a57-8e54-9a82c15e8a2a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:24.687940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:24.689092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:24.690224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 478/958] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:24.695632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631f086-e7a4-4352-9f25-2bcd5820e691 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:24.749795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:24.812126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b631f086-e7a4-4352-9f25-2bcd5820e691 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:24.852187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9fb8a-dd85-4668-9b70-5c1bf7a8e807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:24.852953 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9fb8a-dd85-4668-9b70-5c1bf7a8e807 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:24.854614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9fb8a-dd85-4668-9b70-5c1bf7a8e807 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:24.854614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9fb8a-dd85-4668-9b70-5c1bf7a8e807 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:24.854614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9fb8a-dd85-4668-9b70-5c1bf7a8e807 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:24.854614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9fb8a-dd85-4668-9b70-5c1bf7a8e807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:24.857232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9fb8a-dd85-4668-9b70-5c1bf7a8e807 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:24.862020 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 481/959] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:23:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:24.867945 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342b226-f6fa-4a57-8e54-9a82c15e8a2a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:24.934163 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:24.979058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342b226-f6fa-4a57-8e54-9a82c15e8a2a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:25.013910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ef3d0-4477-4d99-8790-4171378ae1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:25.014900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ef3d0-4477-4d99-8790-4171378ae1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:25.015342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ef3d0-4477-4d99-8790-4171378ae1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:25.015779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ef3d0-4477-4d99-8790-4171378ae1e6 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:25.016642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ef3d0-4477-4d99-8790-4171378ae1e6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:25.017126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ef3d0-4477-4d99-8790-4171378ae1e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:25.018672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ef3d0-4477-4d99-8790-4171378ae1e6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:25.030872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 479/960] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:23:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:25.084028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f26bf-fc4f-4298-8f1c-ff5ac0d72ab6 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:25.084598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f26bf-fc4f-4298-8f1c-ff5ac0d72ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:25.085023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f26bf-fc4f-4298-8f1c-ff5ac0d72ab6 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:25.086993 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 482/961] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:25.096435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4be8dc-8c0a-4341-827f-286c7700460f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:25.096763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4be8dc-8c0a-4341-827f-286c7700460f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:25.096993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4be8dc-8c0a-4341-827f-286c7700460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:25.115218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3d4be8dc-8c0a-4341-827f-286c7700460f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:25.116358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3d4be8dc-8c0a-4341-827f-286c7700460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:25.190795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 480/962] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:23:25.216330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f26bf-fc4f-4298-8f1c-ff5ac0d72ab6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:25.327112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f26bf-fc4f-4298-8f1c-ff5ac0d72ab6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:25.341655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:25.364192 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50579172-9daf-43e6-8414-840951532fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:25.364877 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50579172-9daf-43e6-8414-840951532fc7 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:25.365160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50579172-9daf-43e6-8414-840951532fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:25.365492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50579172-9daf-43e6-8414-840951532fc7 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:25.366158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50579172-9daf-43e6-8414-840951532fc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:25.367644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50579172-9daf-43e6-8414-840951532fc7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:25.368975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: WARNING keystone.server.flask.common [None req-50579172-9daf-43e6-8414-840951532fc7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 29 08:23:25.382718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.sql.core [None req-50579172-9daf-43e6-8414-840951532fc7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 08:23:25.382718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 29 08:23:25.382718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [parameters: {'user_id': 'c236ff813ad34e719f46538a5a2943f4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 29 08:23:25.382718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80131) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-50579172-9daf-43e6-8414-840951532fc7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jun 29 08:23:25.399969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 08:23:25.400915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application yield resource Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application next(self.gen) Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise self.value Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application session.commit() Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 08:23:25.401864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.session.flush() Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application compat.raise_( Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 08:23:25.402785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 08:23:25.403668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application conn.query(q) Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application result.read() Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 08:23:25.404502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c236ff813ad34e719f46538a5a2943f4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:25.405326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 29 08:23:25.406100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 29 08:23:25.412745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 29 08:23:25.412745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 29 08:23:25.412745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 08:23:25.412745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 08:23:25.412745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 29 08:23:25.412745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:25.412745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 483/963] 10.222.0.27 () {60 vars in 1131 bytes} [Wed Jun 29 08:23:25 2022] POST /identity/v3/users => generated 160 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:25.413642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4be8dc-8c0a-4341-827f-286c7700460f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:25.463078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:25.515879 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4be8dc-8c0a-4341-827f-286c7700460f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:25.527988 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abd60-50c5-4acd-ac47-d3f7b24deaa5 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:25.528347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abd60-50c5-4acd-ac47-d3f7b24deaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:25.528630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611abd60-50c5-4acd-ac47-d3f7b24deaa5 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:25.533302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 484/964] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:25.539260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe0755-b57c-422d-834e-bbfa87ea20fd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:25.539545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe0755-b57c-422d-834e-bbfa87ea20fd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:25.539822 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe0755-b57c-422d-834e-bbfa87ea20fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:25.558727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acfe5b-aa80-4bdf-aad3-c647307b3564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:25.559988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acfe5b-aa80-4bdf-aad3-c647307b3564 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:25.560495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acfe5b-aa80-4bdf-aad3-c647307b3564 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:25.561023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acfe5b-aa80-4bdf-aad3-c647307b3564 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:25.561749 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8ebe0755-b57c-422d-834e-bbfa87ea20fd None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:25.563450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8ebe0755-b57c-422d-834e-bbfa87ea20fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:25.569547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acfe5b-aa80-4bdf-aad3-c647307b3564 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:25.571267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acfe5b-aa80-4bdf-aad3-c647307b3564 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-89acfe5b-aa80-4bdf-aad3-c647307b3564 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:25.592332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:25.594145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:25.595984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:25.595984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:25.595984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:25.595984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:25.595984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:25.595984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:25.595984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:25.595984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:25.595984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 08:23:25.595984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:25.603778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 481/965] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:23:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:25.605184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acfe5b-aa80-4bdf-aad3-c647307b3564 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:25.642798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 485/966] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:25.660907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe0755-b57c-422d-834e-bbfa87ea20fd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:25.721398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acfe5b-aa80-4bdf-aad3-c647307b3564 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:25.736705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:25.761230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa649bd4-940f-4026-b814-6870ff39473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:25.762356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649bd4-940f-4026-b814-6870ff39473f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:25.762892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649bd4-940f-4026-b814-6870ff39473f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:25.763363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa649bd4-940f-4026-b814-6870ff39473f None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:25.764465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa649bd4-940f-4026-b814-6870ff39473f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:25.765082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa649bd4-940f-4026-b814-6870ff39473f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:25.766750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa649bd4-940f-4026-b814-6870ff39473f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:25.785128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe0755-b57c-422d-834e-bbfa87ea20fd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:25.787525 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:25.815611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 482/967] 10.222.0.27 () {56 vars in 1101 bytes} [Wed Jun 29 08:23:25 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:25.823943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3762fed-1356-4f3f-af35-15cf8d05d525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:25.824791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762fed-1356-4f3f-af35-15cf8d05d525 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:25.825073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762fed-1356-4f3f-af35-15cf8d05d525 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:25.825343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762fed-1356-4f3f-af35-15cf8d05d525 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:25.839206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3762fed-1356-4f3f-af35-15cf8d05d525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b3762fed-1356-4f3f-af35-15cf8d05d525 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:25.849715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:25.850771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:25.851611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 486/968] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jun 29 08:23:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:25.857304 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa649bd4-940f-4026-b814-6870ff39473f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:25.975389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa649bd4-940f-4026-b814-6870ff39473f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:26.013036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13672146-fea8-4c97-868d-9f1a65b5d058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:26.013719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672146-fea8-4c97-868d-9f1a65b5d058 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:26.014006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672146-fea8-4c97-868d-9f1a65b5d058 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:26.014283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672146-fea8-4c97-868d-9f1a65b5d058 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:26.015075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13672146-fea8-4c97-868d-9f1a65b5d058 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:26.015384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13672146-fea8-4c97-868d-9f1a65b5d058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:26.016768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13672146-fea8-4c97-868d-9f1a65b5d058 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:26.027486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 483/969] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jun 29 08:23:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:26.031683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3762fed-1356-4f3f-af35-15cf8d05d525 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:26.137871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3762fed-1356-4f3f-af35-15cf8d05d525 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:26.175531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e9637-65c1-4179-9d5a-ca6330cd50c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:26.176361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e9637-65c1-4179-9d5a-ca6330cd50c0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:26.176629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e9637-65c1-4179-9d5a-ca6330cd50c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:26.176917 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e9637-65c1-4179-9d5a-ca6330cd50c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:26.188525 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e9637-65c1-4179-9d5a-ca6330cd50c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:26.190561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e9637-65c1-4179-9d5a-ca6330cd50c0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-6c1e9637-65c1-4179-9d5a-ca6330cd50c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:26.207517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:26.209554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:26.211710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:26.211710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:26.211710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:26.211710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:26.213383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 487/970] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:26.215120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13672146-fea8-4c97-868d-9f1a65b5d058 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:26.248260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:26.317770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13672146-fea8-4c97-868d-9f1a65b5d058 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:26.356987 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7680962-2168-4ea7-8a51-9c34f3b0f07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:26.357657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7680962-2168-4ea7-8a51-9c34f3b0f07c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:26.357914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7680962-2168-4ea7-8a51-9c34f3b0f07c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:26.358215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7680962-2168-4ea7-8a51-9c34f3b0f07c None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:26.358977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7680962-2168-4ea7-8a51-9c34f3b0f07c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:26.359264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7680962-2168-4ea7-8a51-9c34f3b0f07c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:26.360664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7680962-2168-4ea7-8a51-9c34f3b0f07c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:26.374795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 484/971] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:26.378531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e9637-65c1-4179-9d5a-ca6330cd50c0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:26.398403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:26.491487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e9637-65c1-4179-9d5a-ca6330cd50c0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:26.526308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee523c9-b91d-415e-86f2-d61aef7e9c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:26.527082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee523c9-b91d-415e-86f2-d61aef7e9c2a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:26.527362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee523c9-b91d-415e-86f2-d61aef7e9c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:26.527634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee523c9-b91d-415e-86f2-d61aef7e9c2a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:26.535548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee523c9-b91d-415e-86f2-d61aef7e9c2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:26.536986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee523c9-b91d-415e-86f2-d61aef7e9c2a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4ee523c9-b91d-415e-86f2-d61aef7e9c2a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:26.551464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:26.552423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:26.553455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:26.553455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:26.553455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:26.553455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:26.553455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:26.553455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:26.553455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:26.553455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:26.553455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 08:23:26.553455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:26.554758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 488/972] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jun 29 08:23:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:26.558718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7680962-2168-4ea7-8a51-9c34f3b0f07c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:26.659077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:26.669494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7680962-2168-4ea7-8a51-9c34f3b0f07c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:26.703485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee524174-efd9-431f-81e7-b0243e72d8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:26.704168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee524174-efd9-431f-81e7-b0243e72d8ca None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:26.704450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee524174-efd9-431f-81e7-b0243e72d8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:26.704771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee524174-efd9-431f-81e7-b0243e72d8ca None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:26.705999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee524174-efd9-431f-81e7-b0243e72d8ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:26.706310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee524174-efd9-431f-81e7-b0243e72d8ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:26.707789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee524174-efd9-431f-81e7-b0243e72d8ca None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:26.726012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 485/973] 10.222.0.27 () {56 vars in 1149 bytes} [Wed Jun 29 08:23:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:26.731538 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee523c9-b91d-415e-86f2-d61aef7e9c2a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:26.797124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cb9bc-7665-46f4-9930-55f04d1e22f7 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:26.797317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cb9bc-7665-46f4-9930-55f04d1e22f7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:26.797583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306cb9bc-7665-46f4-9930-55f04d1e22f7 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:26.799203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 486/974] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:26.804067 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cae67-78d1-475f-b11e-305b15b81840 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:26.804354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cae67-78d1-475f-b11e-305b15b81840 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:26.804625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cae67-78d1-475f-b11e-305b15b81840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:26.822402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ed1cae67-78d1-475f-b11e-305b15b81840 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:26.822632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:26.823334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ed1cae67-78d1-475f-b11e-305b15b81840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:26.854643 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee523c9-b91d-415e-86f2-d61aef7e9c2a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:26.890593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 487/975] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:23:26.891966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939fee9-f0e2-49f4-8fa6-7cab1e27e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:26.892832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939fee9-f0e2-49f4-8fa6-7cab1e27e764 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:26.893271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939fee9-f0e2-49f4-8fa6-7cab1e27e764 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:26.893685 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939fee9-f0e2-49f4-8fa6-7cab1e27e764 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:26.913212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cae67-78d1-475f-b11e-305b15b81840 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:26.966530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939fee9-f0e2-49f4-8fa6-7cab1e27e764 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:26.968499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939fee9-f0e2-49f4-8fa6-7cab1e27e764 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:26.992134 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:27.011390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cae67-78d1-475f-b11e-305b15b81840 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:27.045424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c778df-ad82-41a2-a34d-8867525cf3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:27.046241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c778df-ad82-41a2-a34d-8867525cf3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.046521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c778df-ad82-41a2-a34d-8867525cf3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.046822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c778df-ad82-41a2-a34d-8867525cf3c2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.047539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c778df-ad82-41a2-a34d-8867525cf3c2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:27.049011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c778df-ad82-41a2-a34d-8867525cf3c2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:27.052153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d8c778df-ad82-41a2-a34d-8867525cf3c2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:27.061548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:27.062563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:27.063657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 488/976] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jun 29 08:23:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:27.069233 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f258e-6b01-4012-a594-4f93dd2ffc52 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.069517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f258e-6b01-4012-a594-4f93dd2ffc52 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.069793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f258e-6b01-4012-a594-4f93dd2ffc52 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.071132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 489/977] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:27.071810 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f258e-6b01-4012-a594-4f93dd2ffc52 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:27.084403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 489/978] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:26 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:23:27.085740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2691b07-0a2e-4180-a287-07f98943425d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.086030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2691b07-0a2e-4180-a287-07f98943425d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.086314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2691b07-0a2e-4180-a287-07f98943425d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.103064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b2691b07-0a2e-4180-a287-07f98943425d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:27.104248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b2691b07-0a2e-4180-a287-07f98943425d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:27.168336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 490/979] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:27.169037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2691b07-0a2e-4180-a287-07f98943425d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:27.177054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f258e-6b01-4012-a594-4f93dd2ffc52 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:27.215645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca072c-d6f4-4589-b5a7-671e6c314e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:27.216315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca072c-d6f4-4589-b5a7-671e6c314e85 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.216601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca072c-d6f4-4589-b5a7-671e6c314e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.216879 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bca072c-d6f4-4589-b5a7-671e6c314e85 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.217597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca072c-d6f4-4589-b5a7-671e6c314e85 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:27.217933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca072c-d6f4-4589-b5a7-671e6c314e85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:27.219342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bca072c-d6f4-4589-b5a7-671e6c314e85 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:27.224634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 490/980] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jun 29 08:23:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:27.225387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca072c-d6f4-4589-b5a7-671e6c314e85 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:27.239849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:27.268646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2691b07-0a2e-4180-a287-07f98943425d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:27.277126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:27.302989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6bdfa-62d8-4c0a-ae14-81912ee3865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:27.303714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6bdfa-62d8-4c0a-ae14-81912ee3865d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.303996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6bdfa-62d8-4c0a-ae14-81912ee3865d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.304270 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6bdfa-62d8-4c0a-ae14-81912ee3865d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.315363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6bdfa-62d8-4c0a-ae14-81912ee3865d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:27.316823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6bdfa-62d8-4c0a-ae14-81912ee3865d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:27.329241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bca072c-d6f4-4589-b5a7-671e6c314e85 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-b1b6bdfa-62d8-4c0a-ae14-81912ee3865d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:27.334488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:27.335600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:27.336595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:27.336595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:27.336595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:27.336595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:27.336595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 491/981] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:27.336942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6bdfa-62d8-4c0a-ae14-81912ee3865d None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:27.365083 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5da726-4e34-4f0e-93f1-83d099841124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:27.365782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5da726-4e34-4f0e-93f1-83d099841124 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.366068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5da726-4e34-4f0e-93f1-83d099841124 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.366341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5da726-4e34-4f0e-93f1-83d099841124 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.367095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5da726-4e34-4f0e-93f1-83d099841124 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:27.368474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5da726-4e34-4f0e-93f1-83d099841124 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:27.382336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 491/982] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:23:27 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:27.383152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5da726-4e34-4f0e-93f1-83d099841124 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:27.450526 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6bdfa-62d8-4c0a-ae14-81912ee3865d None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:27.469105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:27.503769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3dae09-05bd-4c93-843e-3a510ca2691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:27.504817 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dae09-05bd-4c93-843e-3a510ca2691b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.505106 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dae09-05bd-4c93-843e-3a510ca2691b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.505381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dae09-05bd-4c93-843e-3a510ca2691b None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.506111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3dae09-05bd-4c93-843e-3a510ca2691b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:27.506422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3dae09-05bd-4c93-843e-3a510ca2691b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:27.507871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3dae09-05bd-4c93-843e-3a510ca2691b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:27.510842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5da726-4e34-4f0e-93f1-83d099841124 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:27.513513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 492/983] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jun 29 08:23:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:27.519588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3dae09-05bd-4c93-843e-3a510ca2691b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:27.542753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:27.552686 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816b968-18cb-4794-a3d2-b15d92f0b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:27.553454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816b968-18cb-4794-a3d2-b15d92f0b1eb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.553647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816b968-18cb-4794-a3d2-b15d92f0b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.553958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816b968-18cb-4794-a3d2-b15d92f0b1eb None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.554723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816b968-18cb-4794-a3d2-b15d92f0b1eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:27.555009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816b968-18cb-4794-a3d2-b15d92f0b1eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:27.556411 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816b968-18cb-4794-a3d2-b15d92f0b1eb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:27.572707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 492/984] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:27.579140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816b968-18cb-4794-a3d2-b15d92f0b1eb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:27.636862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3dae09-05bd-4c93-843e-3a510ca2691b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:27.671978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cc6f2-5955-4af0-a3ea-7a8a505579ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:27.672695 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cc6f2-5955-4af0-a3ea-7a8a505579ab None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.672977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cc6f2-5955-4af0-a3ea-7a8a505579ab None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.673256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cc6f2-5955-4af0-a3ea-7a8a505579ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.674083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cc6f2-5955-4af0-a3ea-7a8a505579ab None admin] RBAC: Adding query filter params (interface=admin, service_id=fc733a28078d45aa9ce69822860b0e32, region_id=RegionOne) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:27.674397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cc6f2-5955-4af0-a3ea-7a8a505579ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:27.675817 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cc6f2-5955-4af0-a3ea-7a8a505579ab None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:27.685661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 493/985] 10.222.0.27 () {56 vars in 1249 bytes} [Wed Jun 29 08:23:27 2022] GET /identity/v3/endpoints?service_id=fc733a28078d45aa9ce69822860b0e32&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:27.685836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816b968-18cb-4794-a3d2-b15d92f0b1eb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:27.686664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cc6f2-5955-4af0-a3ea-7a8a505579ab None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:27.714551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:27.724072 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434eefa-8780-47ea-9458-8045d032e59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:27.725079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434eefa-8780-47ea-9458-8045d032e59d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.725546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434eefa-8780-47ea-9458-8045d032e59d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.725822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434eefa-8780-47ea-9458-8045d032e59d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.732974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434eefa-8780-47ea-9458-8045d032e59d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:27.734532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434eefa-8780-47ea-9458-8045d032e59d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-b434eefa-8780-47ea-9458-8045d032e59d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:27.748561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:27.749488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:27.750361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:27.750361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:27.750361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:27.750361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:27.750361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:27.750361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:27.750361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:27.750361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:27.754953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 493/986] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:27.756121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434eefa-8780-47ea-9458-8045d032e59d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:27.780066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:27.801731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cc6f2-5955-4af0-a3ea-7a8a505579ab None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:27.836907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb668241-7e8f-4ec9-9f14-28c3c96c19fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:27.837576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb668241-7e8f-4ec9-9f14-28c3c96c19fc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.837853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb668241-7e8f-4ec9-9f14-28c3c96c19fc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.838133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb668241-7e8f-4ec9-9f14-28c3c96c19fc None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.839186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb668241-7e8f-4ec9-9f14-28c3c96c19fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:27.840557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb668241-7e8f-4ec9-9f14-28c3c96c19fc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:27.858718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 494/987] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jun 29 08:23:27 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:23:27.872748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434eefa-8780-47ea-9458-8045d032e59d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:27.913358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83238ada-cf20-41c0-8c7c-b430687284ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:27.914044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83238ada-cf20-41c0-8c7c-b430687284ac None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:27.914328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83238ada-cf20-41c0-8c7c-b430687284ac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:27.914604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83238ada-cf20-41c0-8c7c-b430687284ac None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:27.915454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83238ada-cf20-41c0-8c7c-b430687284ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:27.915802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83238ada-cf20-41c0-8c7c-b430687284ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:27.917295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83238ada-cf20-41c0-8c7c-b430687284ac None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:27.937097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 494/988] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:27.938399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb668241-7e8f-4ec9-9f14-28c3c96c19fc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:28.055274 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb668241-7e8f-4ec9-9f14-28c3c96c19fc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:28.091507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129624d-506c-4b31-a211-195e7ca39ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:28.092074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129624d-506c-4b31-a211-195e7ca39ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:28.092361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129624d-506c-4b31-a211-195e7ca39ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:28.092619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129624d-506c-4b31-a211-195e7ca39ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:28.093458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129624d-506c-4b31-a211-195e7ca39ae2 None admin] RBAC: Adding query filter params (role.id=f21b1b6d7f944363b8107baa2cc893ca, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=f320b56bf7ff4c36bbd9730e688612d5) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:28.093772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129624d-506c-4b31-a211-195e7ca39ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:28.095288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129624d-506c-4b31-a211-195e7ca39ae2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:28.101644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 495/989] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:27 2022] GET /identity/v3/role_assignments?user.id=f320b56bf7ff4c36bbd9730e688612d5&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=f21b1b6d7f944363b8107baa2cc893ca => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:29.417407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0665a-aa50-45bb-9241-c15d9371f6aa None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:29.417968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0665a-aa50-45bb-9241-c15d9371f6aa None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:29.417968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c0665a-aa50-45bb-9241-c15d9371f6aa None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:29.419522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 495/990] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:29.424511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d06949-c7fb-4eee-8e3c-5a337f22d802 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:29.424925 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d06949-c7fb-4eee-8e3c-5a337f22d802 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:29.425100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d06949-c7fb-4eee-8e3c-5a337f22d802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:29.442790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c0d06949-c7fb-4eee-8e3c-5a337f22d802 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:29.443982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c0d06949-c7fb-4eee-8e3c-5a337f22d802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:29.447578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:29.521006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 496/991] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:29.545802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0665a-aa50-45bb-9241-c15d9371f6aa None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:29.603265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:29.659984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c0665a-aa50-45bb-9241-c15d9371f6aa None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:29.695678 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e98fd-befc-437a-aaee-767dc154bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:29.696394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e98fd-befc-437a-aaee-767dc154bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:29.696674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e98fd-befc-437a-aaee-767dc154bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:29.696952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e98fd-befc-437a-aaee-767dc154bdf9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:29.697600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e98fd-befc-437a-aaee-767dc154bdf9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:29.699005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e98fd-befc-437a-aaee-767dc154bdf9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-cf1e98fd-befc-437a-aaee-767dc154bdf9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:29.710763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:29.711744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:29.717794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 496/992] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:29.719978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d06949-c7fb-4eee-8e3c-5a337f22d802 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:29.823106 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d06949-c7fb-4eee-8e3c-5a337f22d802 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:29.858701 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbad10d-7c8c-464a-a08a-f48856e96b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:29.859378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbad10d-7c8c-464a-a08a-f48856e96b6c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:29.859658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbad10d-7c8c-464a-a08a-f48856e96b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:29.859922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbad10d-7c8c-464a-a08a-f48856e96b6c None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:29.860635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbad10d-7c8c-464a-a08a-f48856e96b6c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:29.860984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbad10d-7c8c-464a-a08a-f48856e96b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:29.862353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbad10d-7c8c-464a-a08a-f48856e96b6c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:29.867477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 497/993] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:23:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:29.873032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e98fd-befc-437a-aaee-767dc154bdf9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:29.975920 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235dfc1-7d03-45bd-8aa5-59c8657c2639 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:29.976124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235dfc1-7d03-45bd-8aa5-59c8657c2639 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:29.976394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235dfc1-7d03-45bd-8aa5-59c8657c2639 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:29.977744 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 498/994] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:29.982515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07430a65-eb1b-4f6d-b6db-10235c21206d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:29.982901 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07430a65-eb1b-4f6d-b6db-10235c21206d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:29.983266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07430a65-eb1b-4f6d-b6db-10235c21206d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:29.990614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:29.991130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e98fd-befc-437a-aaee-767dc154bdf9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:30.003790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-07430a65-eb1b-4f6d-b6db-10235c21206d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:30.004951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-07430a65-eb1b-4f6d-b6db-10235c21206d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:30.006950 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:30.032031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f9bf0-58bf-4f24-b714-fd5bfd91d13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:30.033010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9bf0-58bf-4f24-b714-fd5bfd91d13d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.033562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9bf0-58bf-4f24-b714-fd5bfd91d13d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.033851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9bf0-58bf-4f24-b714-fd5bfd91d13d None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.034581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f9bf0-58bf-4f24-b714-fd5bfd91d13d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:30.034937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f9bf0-58bf-4f24-b714-fd5bfd91d13d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:30.036359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f9bf0-58bf-4f24-b714-fd5bfd91d13d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:30.042854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 497/995] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:23:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:30.048804 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f9bf0-58bf-4f24-b714-fd5bfd91d13d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:30.086736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 499/996] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:23:30.106185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07430a65-eb1b-4f6d-b6db-10235c21206d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:30.156585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f9bf0-58bf-4f24-b714-fd5bfd91d13d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:30.190716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ac71a-3ac7-4c20-8b81-0d8a028eb1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:30.191386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ac71a-3ac7-4c20-8b81-0d8a028eb1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.191665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ac71a-3ac7-4c20-8b81-0d8a028eb1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.191940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ac71a-3ac7-4c20-8b81-0d8a028eb1a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.192730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ac71a-3ac7-4c20-8b81-0d8a028eb1a3 None admin] RBAC: Adding query filter params (interface=public, service_id=76aaa83607ec43839ea6ab78e167bbec, region_id=RegionOne) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:30.193061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ac71a-3ac7-4c20-8b81-0d8a028eb1a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:30.194418 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ac71a-3ac7-4c20-8b81-0d8a028eb1a3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:30.199681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 498/997] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jun 29 08:23:30 2022] GET /identity/v3/endpoints?service_id=76aaa83607ec43839ea6ab78e167bbec&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:30.200892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc7da9-3870-4d78-8969-5baef2bfc860 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.201261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc7da9-3870-4d78-8969-5baef2bfc860 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.201431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc7da9-3870-4d78-8969-5baef2bfc860 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.202722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 499/998] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:30.205281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc7da9-3870-4d78-8969-5baef2bfc860 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:30.218517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07430a65-eb1b-4f6d-b6db-10235c21206d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:30.247820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:30.254489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8a41d-0c08-4f72-888c-ec7d46e27dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:30.255331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8a41d-0c08-4f72-888c-ec7d46e27dba None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.255616 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8a41d-0c08-4f72-888c-ec7d46e27dba None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.255884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8a41d-0c08-4f72-888c-ec7d46e27dba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.256546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8a41d-0c08-4f72-888c-ec7d46e27dba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:30.257918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8a41d-0c08-4f72-888c-ec7d46e27dba None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:30.259776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-38f8a41d-0c08-4f72-888c-ec7d46e27dba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:30.271763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:30.272757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:30.274012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 500/999] 10.222.0.27 () {56 vars in 1126 bytes} [Wed Jun 29 08:23:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:30.275061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d849f02-dd76-4a8e-b8a6-cafa74943e5e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.275061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d849f02-dd76-4a8e-b8a6-cafa74943e5e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.275061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d849f02-dd76-4a8e-b8a6-cafa74943e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.291499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3d849f02-dd76-4a8e-b8a6-cafa74943e5e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:30.292695 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3d849f02-dd76-4a8e-b8a6-cafa74943e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:30.310061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc7da9-3870-4d78-8969-5baef2bfc860 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:30.344851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429ac41-66a3-4c0c-951d-dcfdd37ae64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:30.345753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429ac41-66a3-4c0c-951d-dcfdd37ae64f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.346039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429ac41-66a3-4c0c-951d-dcfdd37ae64f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.346313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429ac41-66a3-4c0c-951d-dcfdd37ae64f None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.347043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429ac41-66a3-4c0c-951d-dcfdd37ae64f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:30.348421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429ac41-66a3-4c0c-951d-dcfdd37ae64f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:30.354987 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 500/1000] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jun 29 08:23:30 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:23:30.355760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429ac41-66a3-4c0c-951d-dcfdd37ae64f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:30.370748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 501/1001] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:30.388622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d849f02-dd76-4a8e-b8a6-cafa74943e5e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:30.478865 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429ac41-66a3-4c0c-951d-dcfdd37ae64f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:30.482771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:30.510780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d849f02-dd76-4a8e-b8a6-cafa74943e5e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:30.523197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5147f31-ecf6-4845-8cd5-064c281fa13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:30.524113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5147f31-ecf6-4845-8cd5-064c281fa13d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.525091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5147f31-ecf6-4845-8cd5-064c281fa13d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.525091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5147f31-ecf6-4845-8cd5-064c281fa13d None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.525378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5147f31-ecf6-4845-8cd5-064c281fa13d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:30.525685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5147f31-ecf6-4845-8cd5-064c281fa13d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:30.527091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5147f31-ecf6-4845-8cd5-064c281fa13d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:30.534034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 501/1002] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jun 29 08:23:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:30.540215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5147f31-ecf6-4845-8cd5-064c281fa13d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:30.540989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:30.552837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ad2a8-c760-43af-9447-a43fab7ff012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:30.553644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad2a8-c760-43af-9447-a43fab7ff012 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.553921 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad2a8-c760-43af-9447-a43fab7ff012 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.554197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ad2a8-c760-43af-9447-a43fab7ff012 None admin] PATH_INFO: `/v3/services/image` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.554937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ad2a8-c760-43af-9447-a43fab7ff012 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:30.556320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ad2a8-c760-43af-9447-a43fab7ff012 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-619ad2a8-c760-43af-9447-a43fab7ff012 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:30.568335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:30.569317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:30.570151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 502/1003] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Jun 29 08:23:30 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:30.575341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ad2a8-c760-43af-9447-a43fab7ff012 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:30.649612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5147f31-ecf6-4845-8cd5-064c281fa13d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:30.679334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ad2a8-c760-43af-9447-a43fab7ff012 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:30.686169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9111c-4fdb-46a8-85da-52d195a313c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:30.687286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9111c-4fdb-46a8-85da-52d195a313c5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.687570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9111c-4fdb-46a8-85da-52d195a313c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.687843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9111c-4fdb-46a8-85da-52d195a313c5 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.689004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9111c-4fdb-46a8-85da-52d195a313c5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:30.689344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9111c-4fdb-46a8-85da-52d195a313c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:30.691385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9111c-4fdb-46a8-85da-52d195a313c5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:30.702357 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 502/1004] 10.222.0.27 () {56 vars in 1122 bytes} [Wed Jun 29 08:23:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:30.703455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9111c-4fdb-46a8-85da-52d195a313c5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:30.716877 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5392f0d-bb1e-4c30-b22f-789ca34f5fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:30.717548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5392f0d-bb1e-4c30-b22f-789ca34f5fff None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.717832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5392f0d-bb1e-4c30-b22f-789ca34f5fff None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.718104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5392f0d-bb1e-4c30-b22f-789ca34f5fff None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.719117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5392f0d-bb1e-4c30-b22f-789ca34f5fff None admin] RBAC: Adding query filter params (name=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:30.719117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5392f0d-bb1e-4c30-b22f-789ca34f5fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:30.720588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5392f0d-bb1e-4c30-b22f-789ca34f5fff None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:30.725870 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 503/1005] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:30.731145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5392f0d-bb1e-4c30-b22f-789ca34f5fff None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:30.736588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:30.790877 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:30.807016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9111c-4fdb-46a8-85da-52d195a313c5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:30.839933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5392f0d-bb1e-4c30-b22f-789ca34f5fff None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:30.842809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9b38c-6131-414e-9060-d0b1e0b2a864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:30.843476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9b38c-6131-414e-9060-d0b1e0b2a864 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.843755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9b38c-6131-414e-9060-d0b1e0b2a864 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.844027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9b38c-6131-414e-9060-d0b1e0b2a864 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.844764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9b38c-6131-414e-9060-d0b1e0b2a864 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:30.846173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9b38c-6131-414e-9060-d0b1e0b2a864 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:30.870729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 503/1006] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jun 29 08:23:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:30.891420 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b8de5-c1ca-4ba9-bba1-775c8a087aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:30.892179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b8de5-c1ca-4ba9-bba1-775c8a087aad None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:30.892463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b8de5-c1ca-4ba9-bba1-775c8a087aad None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:30.892736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b8de5-c1ca-4ba9-bba1-775c8a087aad None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:30.893455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b8de5-c1ca-4ba9-bba1-775c8a087aad None admin] RBAC: Adding query filter params (type=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:30.893795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b8de5-c1ca-4ba9-bba1-775c8a087aad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:30.895230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b8de5-c1ca-4ba9-bba1-775c8a087aad None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:30.900703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 504/1007] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:30 2022] GET /identity/v3/services?type=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:32.406169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7bb55-92bf-4463-9fb6-f344df3473a2 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:32.406981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7bb55-92bf-4463-9fb6-f344df3473a2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:32.406981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7bb55-92bf-4463-9fb6-f344df3473a2 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:32.408403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 504/1008] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:32.414127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b64e-9255-4623-87ed-bf7d42aacf3f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:32.414583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b64e-9255-4623-87ed-bf7d42aacf3f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:32.414663 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b64e-9255-4623-87ed-bf7d42aacf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:32.434812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1b09b64e-9255-4623-87ed-bf7d42aacf3f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:32.436004 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1b09b64e-9255-4623-87ed-bf7d42aacf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:32.513738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 505/1009] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:32 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:32.536615 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7bb55-92bf-4463-9fb6-f344df3473a2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:32.634516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:32.646218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7bb55-92bf-4463-9fb6-f344df3473a2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:32.681435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd404c9-51bf-4d57-b3cd-1efe9e78421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:32.682149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd404c9-51bf-4d57-b3cd-1efe9e78421e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:32.682455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd404c9-51bf-4d57-b3cd-1efe9e78421e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:32.682740 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd404c9-51bf-4d57-b3cd-1efe9e78421e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:32.683476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd404c9-51bf-4d57-b3cd-1efe9e78421e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:32.684878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd404c9-51bf-4d57-b3cd-1efe9e78421e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-ccd404c9-51bf-4d57-b3cd-1efe9e78421e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:32.701682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:32.703050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 08:23:32.704006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:32.707133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 505/1010] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:32.710522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b64e-9255-4623-87ed-bf7d42aacf3f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:32.768578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:32.827786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b64e-9255-4623-87ed-bf7d42aacf3f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:32.864259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463bfef-377f-489a-ae41-7aaaa61e7542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:32.864926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463bfef-377f-489a-ae41-7aaaa61e7542 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:32.865210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463bfef-377f-489a-ae41-7aaaa61e7542 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:32.865487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463bfef-377f-489a-ae41-7aaaa61e7542 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:32.866197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463bfef-377f-489a-ae41-7aaaa61e7542 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:32.866504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463bfef-377f-489a-ae41-7aaaa61e7542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:32.867915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463bfef-377f-489a-ae41-7aaaa61e7542 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:32.873039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 506/1011] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:23:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:32.879238 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd404c9-51bf-4d57-b3cd-1efe9e78421e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:32.912849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b69b18-54de-422a-8bfd-08f646d195ca None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:32.913092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b69b18-54de-422a-8bfd-08f646d195ca None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:32.913357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b69b18-54de-422a-8bfd-08f646d195ca None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:32.914770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 507/1012] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:32.918113 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4621097-a539-4ff2-90dc-107f4aded85a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:32.918397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4621097-a539-4ff2-90dc-107f4aded85a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:32.918703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4621097-a539-4ff2-90dc-107f4aded85a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:32.920182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 508/1013] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:32.922094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895bfb7-0745-45fe-b42c-0edbae137d19 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:32.922094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895bfb7-0745-45fe-b42c-0edbae137d19 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:32.922094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895bfb7-0745-45fe-b42c-0edbae137d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:32.942880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9895bfb7-0745-45fe-b42c-0edbae137d19 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:32.944003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9895bfb7-0745-45fe-b42c-0edbae137d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:32.983129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd404c9-51bf-4d57-b3cd-1efe9e78421e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:33.012073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 509/1014] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:32 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:33.013423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa941aad-afa7-44fc-a598-70feef7cbf0f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:33.013711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa941aad-afa7-44fc-a598-70feef7cbf0f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:33.013990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa941aad-afa7-44fc-a598-70feef7cbf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:33.019961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eadcb27-1834-427d-b21b-619cf11290d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:33.020751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadcb27-1834-427d-b21b-619cf11290d8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:33.021043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadcb27-1834-427d-b21b-619cf11290d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:33.021320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadcb27-1834-427d-b21b-619cf11290d8 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:33.022032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eadcb27-1834-427d-b21b-619cf11290d8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:33.022344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eadcb27-1834-427d-b21b-619cf11290d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:33.023752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eadcb27-1834-427d-b21b-619cf11290d8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:33.034188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 506/1015] 10.222.0.27 () {56 vars in 1112 bytes} [Wed Jun 29 08:23:32 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:33.034369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fa941aad-afa7-44fc-a598-70feef7cbf0f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:33.034369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fa941aad-afa7-44fc-a598-70feef7cbf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:33.034946 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eadcb27-1834-427d-b21b-619cf11290d8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:33.059801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:33.067881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:33.106098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 510/1016] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:33 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:33.106954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa941aad-afa7-44fc-a598-70feef7cbf0f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:33.141550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eadcb27-1834-427d-b21b-619cf11290d8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:33.175663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a6c21-46b8-4348-940c-6dd23f928ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:33.176380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a6c21-46b8-4348-940c-6dd23f928ea2 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:33.176722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a6c21-46b8-4348-940c-6dd23f928ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:33.177000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a6c21-46b8-4348-940c-6dd23f928ea2 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:33.177751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a6c21-46b8-4348-940c-6dd23f928ea2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:33.179482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a6c21-46b8-4348-940c-6dd23f928ea2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:33.194337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 507/1017] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:23:33 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:23:33.195109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a6c21-46b8-4348-940c-6dd23f928ea2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:33.217940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa941aad-afa7-44fc-a598-70feef7cbf0f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:33.269256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8dfb2-d2c5-4040-841f-8c89c7c21bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:33.270259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8dfb2-d2c5-4040-841f-8c89c7c21bd7 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:33.270730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8dfb2-d2c5-4040-841f-8c89c7c21bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:33.271207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8dfb2-d2c5-4040-841f-8c89c7c21bd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:33.272267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8dfb2-d2c5-4040-841f-8c89c7c21bd7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:33.273828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8dfb2-d2c5-4040-841f-8c89c7c21bd7 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:33.293548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:33.300394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 511/1018] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jun 29 08:23:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:33.318364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a6c21-46b8-4348-940c-6dd23f928ea2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:33.372895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e38c1-6a63-4e72-8944-db526023f094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:33.373787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e38c1-6a63-4e72-8944-db526023f094 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:33.374070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e38c1-6a63-4e72-8944-db526023f094 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:33.374346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e38c1-6a63-4e72-8944-db526023f094 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:33.377112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e38c1-6a63-4e72-8944-db526023f094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:33.377705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e38c1-6a63-4e72-8944-db526023f094 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:33.379604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 508/1019] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:33.386087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8dfb2-d2c5-4040-841f-8c89c7c21bd7 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:33.416467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:33.512957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8dfb2-d2c5-4040-841f-8c89c7c21bd7 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:33.550543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ded7e-8f8a-48ef-93db-f9f9533f6ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:33.551247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ded7e-8f8a-48ef-93db-f9f9533f6ff3 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:33.551540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ded7e-8f8a-48ef-93db-f9f9533f6ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:33.551810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ded7e-8f8a-48ef-93db-f9f9533f6ff3 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:33.552527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ded7e-8f8a-48ef-93db-f9f9533f6ff3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:33.553974 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ded7e-8f8a-48ef-93db-f9f9533f6ff3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:33.584142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 512/1020] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:23:33 2022] POST /identity/v3/projects => generated 311 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:35.250560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504684d-9710-4352-9d36-7685fab9c569 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.251142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504684d-9710-4352-9d36-7685fab9c569 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.251142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504684d-9710-4352-9d36-7685fab9c569 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.252857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 509/1021] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:35.258108 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11736cf-4e3b-4774-875a-bc4d3885fee5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.258575 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11736cf-4e3b-4774-875a-bc4d3885fee5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.258655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11736cf-4e3b-4774-875a-bc4d3885fee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.275951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d11736cf-4e3b-4774-875a-bc4d3885fee5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:35.277141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d11736cf-4e3b-4774-875a-bc4d3885fee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:35.350197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 513/1022] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:35.373612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504684d-9710-4352-9d36-7685fab9c569 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:35.436311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081e2aa-e1c5-49be-99a5-4bce36a12201 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.436578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081e2aa-e1c5-49be-99a5-4bce36a12201 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.436854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a081e2aa-e1c5-49be-99a5-4bce36a12201 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.438501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 514/1023] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:35.443705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d3fbe-0767-444f-955e-9a2c47992e76 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.443996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d3fbe-0767-444f-955e-9a2c47992e76 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.444271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d3fbe-0767-444f-955e-9a2c47992e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.451072 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:35.470823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0a1d3fbe-0767-444f-955e-9a2c47992e76 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:35.472031 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0a1d3fbe-0767-444f-955e-9a2c47992e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:35.492298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504684d-9710-4352-9d36-7685fab9c569 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:35.536300 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1625c45-5f3c-4e64-be58-c71957ce2015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:35.537132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1625c45-5f3c-4e64-be58-c71957ce2015 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.537392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1625c45-5f3c-4e64-be58-c71957ce2015 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.537698 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1625c45-5f3c-4e64-be58-c71957ce2015 None admin] PATH_INFO: `/v3/services/image` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.538361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1625c45-5f3c-4e64-be58-c71957ce2015 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:35.539835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1625c45-5f3c-4e64-be58-c71957ce2015 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:35.540461 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 515/1024] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f1625c45-5f3c-4e64-be58-c71957ce2015 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:35.556336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:35.557349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 08:23:35.558186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:35.559720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 510/1025] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Jun 29 08:23:35 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:35.563518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d3fbe-0767-444f-955e-9a2c47992e76 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:35.563647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1625c45-5f3c-4e64-be58-c71957ce2015 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:35.575736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:35.666284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1625c45-5f3c-4e64-be58-c71957ce2015 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:35.666596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d3fbe-0767-444f-955e-9a2c47992e76 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:35.699679 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7180eab-3dad-4d40-afe2-4875b66ee7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:35.699865 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becddef-965f-40d3-8016-81f5ec563299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:35.700342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7180eab-3dad-4d40-afe2-4875b66ee7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.700511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becddef-965f-40d3-8016-81f5ec563299 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.700647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7180eab-3dad-4d40-afe2-4875b66ee7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.700794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becddef-965f-40d3-8016-81f5ec563299 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.700895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7180eab-3dad-4d40-afe2-4875b66ee7a0 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.701067 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becddef-965f-40d3-8016-81f5ec563299 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.701591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7180eab-3dad-4d40-afe2-4875b66ee7a0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:35.701904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7180eab-3dad-4d40-afe2-4875b66ee7a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:35.703319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7180eab-3dad-4d40-afe2-4875b66ee7a0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:35.704370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:35.709435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 516/1026] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:35.710675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0baa65-7b4a-42bd-936b-876aa7d67e2e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.710989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0baa65-7b4a-42bd-936b-876aa7d67e2e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.711268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0baa65-7b4a-42bd-936b-876aa7d67e2e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.711424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becddef-965f-40d3-8016-81f5ec563299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:35.714741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 517/1027] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:35.715194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0baa65-7b4a-42bd-936b-876aa7d67e2e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-1becddef-965f-40d3-8016-81f5ec563299 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:35.720311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:35.721264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:35.722184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 511/1028] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jun 29 08:23:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:35.722929 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c790c5-ad29-45d5-8f66-9f69afb7ce77 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.723492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c790c5-ad29-45d5-8f66-9f69afb7ce77 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.723492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c790c5-ad29-45d5-8f66-9f69afb7ce77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.739732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d5c790c5-ad29-45d5-8f66-9f69afb7ce77 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:35.740920 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d5c790c5-ad29-45d5-8f66-9f69afb7ce77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:35.803955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:35.810037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 512/1029] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:35.810836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c790c5-ad29-45d5-8f66-9f69afb7ce77 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:35.815986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0baa65-7b4a-42bd-936b-876aa7d67e2e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:35.850958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6a035-6a51-45c1-8ab1-85c54db10f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:35.851619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a035-6a51-45c1-8ab1-85c54db10f10 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.851898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a035-6a51-45c1-8ab1-85c54db10f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.852166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6a035-6a51-45c1-8ab1-85c54db10f10 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.852869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6a035-6a51-45c1-8ab1-85c54db10f10 None admin] RBAC: Adding query filter params (type=image) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:35.853171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6a035-6a51-45c1-8ab1-85c54db10f10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:35.854535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6a035-6a51-45c1-8ab1-85c54db10f10 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:35.859250 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 518/1030] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:35.860002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6a035-6a51-45c1-8ab1-85c54db10f10 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:35.914614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c790c5-ad29-45d5-8f66-9f69afb7ce77 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:35.918807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:35.949464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29b54a-519b-4242-945e-6bee388b2a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:35.949956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29b54a-519b-4242-945e-6bee388b2a7a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.950245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29b54a-519b-4242-945e-6bee388b2a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.950583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29b54a-519b-4242-945e-6bee388b2a7a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.951306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29b54a-519b-4242-945e-6bee388b2a7a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:35.951621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29b54a-519b-4242-945e-6bee388b2a7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:35.952968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29b54a-519b-4242-945e-6bee388b2a7a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:35.958122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6a035-6a51-45c1-8ab1-85c54db10f10 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:35.961427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 513/1031] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jun 29 08:23:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:35.962210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29b54a-519b-4242-945e-6bee388b2a7a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:35.994153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46303bbc-7d0b-413a-be32-fe311d1763ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:35.994977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303bbc-7d0b-413a-be32-fe311d1763ce None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:35.995256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303bbc-7d0b-413a-be32-fe311d1763ce None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:35.995524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46303bbc-7d0b-413a-be32-fe311d1763ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:35.996714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46303bbc-7d0b-413a-be32-fe311d1763ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:35.998137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46303bbc-7d0b-413a-be32-fe311d1763ce None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:35.999779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 519/1032] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:36.000523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46303bbc-7d0b-413a-be32-fe311d1763ce None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:36.023929 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:36.064482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29b54a-519b-4242-945e-6bee388b2a7a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:36.099698 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2342d-7787-4bee-8a34-7e0e5e6e59f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:36.099894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46303bbc-7d0b-413a-be32-fe311d1763ce None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:36.100365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2342d-7787-4bee-8a34-7e0e5e6e59f8 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:36.100649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2342d-7787-4bee-8a34-7e0e5e6e59f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:36.100924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af2342d-7787-4bee-8a34-7e0e5e6e59f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:36.101721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2342d-7787-4bee-8a34-7e0e5e6e59f8 None admin] RBAC: Adding query filter params (interface=public, service_id=0618de34530a4ba5ac68f3c2815e6507, region_id=RegionOne) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:36.102022 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2342d-7787-4bee-8a34-7e0e5e6e59f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:36.103425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af2342d-7787-4bee-8a34-7e0e5e6e59f8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:36.108437 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 514/1033] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jun 29 08:23:35 2022] GET /identity/v3/endpoints?service_id=0618de34530a4ba5ac68f3c2815e6507&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:36.109201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2342d-7787-4bee-8a34-7e0e5e6e59f8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:36.127741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:36.134333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24912018-ce83-407e-937f-d4f5cc1b4a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:36.135163 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24912018-ce83-407e-937f-d4f5cc1b4a50 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:36.135443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24912018-ce83-407e-937f-d4f5cc1b4a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:36.135716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24912018-ce83-407e-937f-d4f5cc1b4a50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:36.146298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24912018-ce83-407e-937f-d4f5cc1b4a50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:36.147814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24912018-ce83-407e-937f-d4f5cc1b4a50 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-24912018-ce83-407e-937f-d4f5cc1b4a50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:36.164552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:36.165557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:36.166468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:36.166468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:36.166468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:36.166468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:36.166468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 520/1034] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:36.166764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24912018-ce83-407e-937f-d4f5cc1b4a50 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:36.215787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af2342d-7787-4bee-8a34-7e0e5e6e59f8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:36.238271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:36.249307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a63ab-ee32-4a15-8cdd-497e16357dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:36.249977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a63ab-ee32-4a15-8cdd-497e16357dc8 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:36.250256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a63ab-ee32-4a15-8cdd-497e16357dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:36.250530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a63ab-ee32-4a15-8cdd-497e16357dc8 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:36.251314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a63ab-ee32-4a15-8cdd-497e16357dc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:36.252755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a63ab-ee32-4a15-8cdd-497e16357dc8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:36.268185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24912018-ce83-407e-937f-d4f5cc1b4a50 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:36.277803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 515/1035] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:23:36 2022] POST /identity/v3/users => generated 285 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:36.278649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a63ab-ee32-4a15-8cdd-497e16357dc8 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:36.320352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935dc43-c617-4f64-aa9a-863629eae7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:36.320943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935dc43-c617-4f64-aa9a-863629eae7ad None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:36.321223 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935dc43-c617-4f64-aa9a-863629eae7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:36.321494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935dc43-c617-4f64-aa9a-863629eae7ad None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:36.322176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935dc43-c617-4f64-aa9a-863629eae7ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:36.323605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935dc43-c617-4f64-aa9a-863629eae7ad None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:36.332373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 521/1036] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jun 29 08:23:36 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:23:36.396712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a63ab-ee32-4a15-8cdd-497e16357dc8 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:36.437166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cefcd1-4131-4723-95f1-eeae276429d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:36.437866 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cefcd1-4131-4723-95f1-eeae276429d6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:36.438197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cefcd1-4131-4723-95f1-eeae276429d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:36.438475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cefcd1-4131-4723-95f1-eeae276429d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:36.439541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cefcd1-4131-4723-95f1-eeae276429d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:36.439857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cefcd1-4131-4723-95f1-eeae276429d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:36.441263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cefcd1-4131-4723-95f1-eeae276429d6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:36.456439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 516/1037] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:36.464976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935dc43-c617-4f64-aa9a-863629eae7ad None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:36.518941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:36.573222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935dc43-c617-4f64-aa9a-863629eae7ad None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:36.607998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56480c4-366d-4cd7-9ca3-2a4c44b37092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:36.608793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56480c4-366d-4cd7-9ca3-2a4c44b37092 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:36.609076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56480c4-366d-4cd7-9ca3-2a4c44b37092 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:36.609338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56480c4-366d-4cd7-9ca3-2a4c44b37092 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:36.617313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56480c4-366d-4cd7-9ca3-2a4c44b37092 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:36.618787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56480c4-366d-4cd7-9ca3-2a4c44b37092 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c56480c4-366d-4cd7-9ca3-2a4c44b37092 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:36.633237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:36.634172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:36.635138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:36.635138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:36.635138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:36.635138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:36.635138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:36.635138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:36.635138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:36.635138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:36.635138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:23:36.635138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:36.635138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 522/1038] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jun 29 08:23:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:36.640642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cefcd1-4131-4723-95f1-eeae276429d6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:36.685306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:36.747549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cefcd1-4131-4723-95f1-eeae276429d6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:36.784856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ee7d-80e0-481d-904e-fdd0df6ad1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:36.785535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ee7d-80e0-481d-904e-fdd0df6ad1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:36.785812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ee7d-80e0-481d-904e-fdd0df6ad1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:36.786085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0619ee7d-80e0-481d-904e-fdd0df6ad1f6 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:36.786952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619ee7d-80e0-481d-904e-fdd0df6ad1f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:36.787263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619ee7d-80e0-481d-904e-fdd0df6ad1f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:36.788703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0619ee7d-80e0-481d-904e-fdd0df6ad1f6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:36.807440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 517/1039] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jun 29 08:23:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:36.813013 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56480c4-366d-4cd7-9ca3-2a4c44b37092 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:36.917020 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56480c4-366d-4cd7-9ca3-2a4c44b37092 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:36.924581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:36.952426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68df468-3c19-4429-a192-58587f6142ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:36.953070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68df468-3c19-4429-a192-58587f6142ab None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:36.953346 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68df468-3c19-4429-a192-58587f6142ab None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:36.953618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68df468-3c19-4429-a192-58587f6142ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:37.022892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68df468-3c19-4429-a192-58587f6142ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:37.024347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68df468-3c19-4429-a192-58587f6142ab None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:37.124812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 523/1040] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:36 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:23:37.130509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ee7d-80e0-481d-904e-fdd0df6ad1f6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:37.228093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:37.232623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0619ee7d-80e0-481d-904e-fdd0df6ad1f6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:37.266259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e2bc0-04ff-476b-9845-ac32b968707d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:37.267118 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e2bc0-04ff-476b-9845-ac32b968707d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:37.267402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e2bc0-04ff-476b-9845-ac32b968707d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:37.267672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e2bc0-04ff-476b-9845-ac32b968707d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:37.277571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e2bc0-04ff-476b-9845-ac32b968707d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:37.279035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e2bc0-04ff-476b-9845-ac32b968707d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-7f3e2bc0-04ff-476b-9845-ac32b968707d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:37.296179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:37.297075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:37.297975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:37.297975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:37.297975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:37.297975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:37.299000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 518/1041] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:37.303197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68df468-3c19-4429-a192-58587f6142ab None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:37.311786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:37.403938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68df468-3c19-4429-a192-58587f6142ab None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:37.438316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba69231-e4e6-4eb0-b395-3cf90b2bef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:37.439091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba69231-e4e6-4eb0-b395-3cf90b2bef35 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:37.439370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba69231-e4e6-4eb0-b395-3cf90b2bef35 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:37.439639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba69231-e4e6-4eb0-b395-3cf90b2bef35 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:37.440383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba69231-e4e6-4eb0-b395-3cf90b2bef35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:37.440692 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba69231-e4e6-4eb0-b395-3cf90b2bef35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:37.442133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba69231-e4e6-4eb0-b395-3cf90b2bef35 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:37.464200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 524/1042] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:37.470218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e2bc0-04ff-476b-9845-ac32b968707d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:37.572194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e2bc0-04ff-476b-9845-ac32b968707d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:37.606050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe38159-b906-4c07-8058-24c4feb3ddd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:37.606868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe38159-b906-4c07-8058-24c4feb3ddd9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:37.607145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe38159-b906-4c07-8058-24c4feb3ddd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:37.607414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe38159-b906-4c07-8058-24c4feb3ddd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:37.614054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe38159-b906-4c07-8058-24c4feb3ddd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:37.615517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe38159-b906-4c07-8058-24c4feb3ddd9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-abe38159-b906-4c07-8058-24c4feb3ddd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:37.630099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:37.631074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:37.637149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:37.637149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:37.637149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:37.637149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:37.637149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:37.637149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:37.637149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:37.637149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:37.637149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 519/1043] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:37.637651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba69231-e4e6-4eb0-b395-3cf90b2bef35 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:37.704966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:37.740424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba69231-e4e6-4eb0-b395-3cf90b2bef35 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:37.773080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8bfa6-7df3-405c-aafa-050861f8a506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:37.773734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8bfa6-7df3-405c-aafa-050861f8a506 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:37.774017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8bfa6-7df3-405c-aafa-050861f8a506 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:37.774289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8bfa6-7df3-405c-aafa-050861f8a506 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:37.775102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8bfa6-7df3-405c-aafa-050861f8a506 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:37.775407 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8bfa6-7df3-405c-aafa-050861f8a506 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:37.776827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8bfa6-7df3-405c-aafa-050861f8a506 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:37.789056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 525/1044] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:37.794956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe38159-b906-4c07-8058-24c4feb3ddd9 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:37.808154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:37.897185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe38159-b906-4c07-8058-24c4feb3ddd9 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:37.930809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026899e0-5793-4f0a-a36a-184eff1809b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:37.931477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026899e0-5793-4f0a-a36a-184eff1809b2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:37.931742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026899e0-5793-4f0a-a36a-184eff1809b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:37.932011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026899e0-5793-4f0a-a36a-184eff1809b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:37.932846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026899e0-5793-4f0a-a36a-184eff1809b2 None admin] RBAC: Adding query filter params (role.id=f21b1b6d7f944363b8107baa2cc893ca, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=7bd500f020f143339c5a4fc9f0728ee4) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:37.933157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026899e0-5793-4f0a-a36a-184eff1809b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:37.934595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026899e0-5793-4f0a-a36a-184eff1809b2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:37.940447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 520/1045] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:37 2022] GET /identity/v3/role_assignments?user.id=7bd500f020f143339c5a4fc9f0728ee4&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=f21b1b6d7f944363b8107baa2cc893ca => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:38.359575 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e8afb-e080-4662-abbb-bf6879fb612a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:38.360130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e8afb-e080-4662-abbb-bf6879fb612a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:38.360130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e8afb-e080-4662-abbb-bf6879fb612a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:38.361869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 526/1046] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:38.367042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd67f-b904-4ec5-a875-3ca8244459d6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:38.367468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd67f-b904-4ec5-a875-3ca8244459d6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:38.367544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd67f-b904-4ec5-a875-3ca8244459d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:38.384923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f6acd67f-b904-4ec5-a875-3ca8244459d6 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:38.386103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f6acd67f-b904-4ec5-a875-3ca8244459d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:38.389796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670676a-de0c-4502-b96c-d4dcacb85004 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:38.389796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670676a-de0c-4502-b96c-d4dcacb85004 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:38.389796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670676a-de0c-4502-b96c-d4dcacb85004 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:38.391282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 527/1047] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:38.398787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd2ce0-3116-4bcf-b062-d5fc5ee5e1b2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:38.399082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd2ce0-3116-4bcf-b062-d5fc5ee5e1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:38.399649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd2ce0-3116-4bcf-b062-d5fc5ee5e1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:38.418113 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b4fd2ce0-3116-4bcf-b062-d5fc5ee5e1b2 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:38.419335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b4fd2ce0-3116-4bcf-b062-d5fc5ee5e1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:38.466466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 521/1048] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:38.489544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acd67f-b904-4ec5-a875-3ca8244459d6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:38.493423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:38.499033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 528/1049] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:38.515462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd2ce0-3116-4bcf-b062-d5fc5ee5e1b2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:38.578544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:38.598095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acd67f-b904-4ec5-a875-3ca8244459d6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:38.624618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd2ce0-3116-4bcf-b062-d5fc5ee5e1b2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:38.633368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc161c-3f63-4447-91ee-4a62d59d5f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:38.634203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc161c-3f63-4447-91ee-4a62d59d5f46 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:38.634717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc161c-3f63-4447-91ee-4a62d59d5f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:38.634923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc161c-3f63-4447-91ee-4a62d59d5f46 None admin] PATH_INFO: `/v3/services/image` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:38.635527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc161c-3f63-4447-91ee-4a62d59d5f46 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:38.637028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bc161c-3f63-4447-91ee-4a62d59d5f46 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-56bc161c-3f63-4447-91ee-4a62d59d5f46 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:38.649616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:38.650642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:38.651648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 522/1050] 10.222.0.27 () {56 vars in 1105 bytes} [Wed Jun 29 08:23:38 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:38.656994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc161c-3f63-4447-91ee-4a62d59d5f46 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:38.662402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0495d-f859-4ef6-afb0-6e076b144812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:38.663231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0495d-f859-4ef6-afb0-6e076b144812 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:38.663513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0495d-f859-4ef6-afb0-6e076b144812 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:38.663786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0495d-f859-4ef6-afb0-6e076b144812 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:38.674200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b0495d-f859-4ef6-afb0-6e076b144812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-43b0495d-f859-4ef6-afb0-6e076b144812 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:38.681819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:38.682963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:38.683848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 529/1051] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:23:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:38.688904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0495d-f859-4ef6-afb0-6e076b144812 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:38.727734 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:38.764198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc161c-3f63-4447-91ee-4a62d59d5f46 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:38.791935 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0495d-f859-4ef6-afb0-6e076b144812 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:38.800346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138572e-9937-4f8c-9e88-c3939b3eb384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:38.801021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138572e-9937-4f8c-9e88-c3939b3eb384 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:38.801303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138572e-9937-4f8c-9e88-c3939b3eb384 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:38.801579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138572e-9937-4f8c-9e88-c3939b3eb384 None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:38.802291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138572e-9937-4f8c-9e88-c3939b3eb384 None admin] RBAC: Adding query filter params (name=image) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:38.802600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138572e-9937-4f8c-9e88-c3939b3eb384 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:38.803998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138572e-9937-4f8c-9e88-c3939b3eb384 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:38.808664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:38.809436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 523/1052] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:38.815222 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138572e-9937-4f8c-9e88-c3939b3eb384 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:38.832725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ca45e-881a-418e-b86c-ca363b09624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:38.833480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ca45e-881a-418e-b86c-ca363b09624a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:38.833762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ca45e-881a-418e-b86c-ca363b09624a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:38.834036 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ca45e-881a-418e-b86c-ca363b09624a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:38.834809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ca45e-881a-418e-b86c-ca363b09624a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:38.835125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ca45e-881a-418e-b86c-ca363b09624a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:38.836516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ca45e-881a-418e-b86c-ca363b09624a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:38.851158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 530/1053] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:38.852336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ca45e-881a-418e-b86c-ca363b09624a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:38.926330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138572e-9937-4f8c-9e88-c3939b3eb384 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:38.947976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ca45e-881a-418e-b86c-ca363b09624a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:38.956904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:38.963376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743f921-0e46-424b-9c96-4838e72dd46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:38.964583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743f921-0e46-424b-9c96-4838e72dd46d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:38.965024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743f921-0e46-424b-9c96-4838e72dd46d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:38.965450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743f921-0e46-424b-9c96-4838e72dd46d None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:38.966415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743f921-0e46-424b-9c96-4838e72dd46d None admin] RBAC: Adding query filter params (type=image) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:38.966940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743f921-0e46-424b-9c96-4838e72dd46d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:38.968450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743f921-0e46-424b-9c96-4838e72dd46d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:38.973988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 524/1054] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:38.979635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743f921-0e46-424b-9c96-4838e72dd46d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:38.987623 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e093ad-33de-4fa3-b259-125b903cf464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:38.988494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e093ad-33de-4fa3-b259-125b903cf464 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:38.988768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e093ad-33de-4fa3-b259-125b903cf464 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:38.989044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e093ad-33de-4fa3-b259-125b903cf464 None admin] PATH_INFO: `/v3/users/2d8827247f0847178f2204329595ce17` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:39.003466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e093ad-33de-4fa3-b259-125b903cf464 None admin] RBAC: Authorizing `identity:get_user(user_id=2d8827247f0847178f2204329595ce17)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:39.004947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e093ad-33de-4fa3-b259-125b903cf464 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:39.006676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 531/1055] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:23:38 2022] GET /identity/v3/users/2d8827247f0847178f2204329595ce17 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:39.012362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e093ad-33de-4fa3-b259-125b903cf464 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:39.050281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:39.093812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743f921-0e46-424b-9c96-4838e72dd46d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:39.123395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e093ad-33de-4fa3-b259-125b903cf464 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:39.128916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57998709-36f4-4db5-bae6-ef687a6ed78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:39.129583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57998709-36f4-4db5-bae6-ef687a6ed78a None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:39.129868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57998709-36f4-4db5-bae6-ef687a6ed78a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:39.130145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57998709-36f4-4db5-bae6-ef687a6ed78a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:39.130914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57998709-36f4-4db5-bae6-ef687a6ed78a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:39.132295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57998709-36f4-4db5-bae6-ef687a6ed78a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:39.154242 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 525/1056] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jun 29 08:23:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:39.173218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd245de8-3cd5-487e-b5e8-980442ae6a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:39.173887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd245de8-3cd5-487e-b5e8-980442ae6a28 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:39.174166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd245de8-3cd5-487e-b5e8-980442ae6a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:39.174435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd245de8-3cd5-487e-b5e8-980442ae6a28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:39.251620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd245de8-3cd5-487e-b5e8-980442ae6a28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:39.253177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd245de8-3cd5-487e-b5e8-980442ae6a28 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:39.284915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:39.361700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 532/1057] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 08:23:39.367734 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57998709-36f4-4db5-bae6-ef687a6ed78a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:39.422361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:39.480537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57998709-36f4-4db5-bae6-ef687a6ed78a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:39.517611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3bb17-3cc7-4ff5-b7db-b5e930ca299d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:39.518332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3bb17-3cc7-4ff5-b7db-b5e930ca299d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:39.518608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3bb17-3cc7-4ff5-b7db-b5e930ca299d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:39.518909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3bb17-3cc7-4ff5-b7db-b5e930ca299d None admin] PATH_INFO: `/v3/projects/baa5396629c841728e8186c551bf073a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:39.520131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3bb17-3cc7-4ff5-b7db-b5e930ca299d None admin] RBAC: Authorizing `identity:get_project(project_id=baa5396629c841728e8186c551bf073a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:39.521573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3bb17-3cc7-4ff5-b7db-b5e930ca299d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:39.523498 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 526/1058] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:23:39 2022] GET /identity/v3/projects/baa5396629c841728e8186c551bf073a => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:39.529650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd245de8-3cd5-487e-b5e8-980442ae6a28 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:39.633913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd245de8-3cd5-487e-b5e8-980442ae6a28 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:39.664191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:39.670432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb2e5e-e551-450d-bb99-69f1b8c43463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:39.671200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb2e5e-e551-450d-bb99-69f1b8c43463 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:39.671485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb2e5e-e551-450d-bb99-69f1b8c43463 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:39.671771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb2e5e-e551-450d-bb99-69f1b8c43463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:39.672613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb2e5e-e551-450d-bb99-69f1b8c43463 None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=baa5396629c841728e8186c551bf073a, user.id=2d8827247f0847178f2204329595ce17) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:39.672930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb2e5e-e551-450d-bb99-69f1b8c43463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:39.674463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb2e5e-e551-450d-bb99-69f1b8c43463 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:39.680743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 533/1059] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:39 2022] GET /identity/v3/role_assignments?user.id=2d8827247f0847178f2204329595ce17&scope.project.id=baa5396629c841728e8186c551bf073a&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:39.973620 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef44f4a-93c9-4f55-8504-24d9f68bb201 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:39.973992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef44f4a-93c9-4f55-8504-24d9f68bb201 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:39.974177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef44f4a-93c9-4f55-8504-24d9f68bb201 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:39.975739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 527/1060] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:39.980808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955df3e7-bc79-448d-9885-a9361e6b55ef None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:39.981103 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955df3e7-bc79-448d-9885-a9361e6b55ef None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:39.981376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955df3e7-bc79-448d-9885-a9361e6b55ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:39.998803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-955df3e7-bc79-448d-9885-a9361e6b55ef None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:39.999970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-955df3e7-bc79-448d-9885-a9361e6b55ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:40.068889 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 534/1061] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:40.091381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef44f4a-93c9-4f55-8504-24d9f68bb201 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:40.201200 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef44f4a-93c9-4f55-8504-24d9f68bb201 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:40.222003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:40.237131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298964fa-b36f-4bc0-a693-de582c8e2895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:40.237875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298964fa-b36f-4bc0-a693-de582c8e2895 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:40.238153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298964fa-b36f-4bc0-a693-de582c8e2895 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:40.238421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298964fa-b36f-4bc0-a693-de582c8e2895 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:40.248643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298964fa-b36f-4bc0-a693-de582c8e2895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-298964fa-b36f-4bc0-a693-de582c8e2895 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:40.256703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:40.257640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:40.258469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:40.266104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 528/1062] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jun 29 08:23:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:40.266241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955df3e7-bc79-448d-9885-a9361e6b55ef None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:40.371961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955df3e7-bc79-448d-9885-a9361e6b55ef None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:40.398215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:40.408557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ccaef-f225-415a-8353-43a49b96f06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:40.409229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ccaef-f225-415a-8353-43a49b96f06b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:40.409510 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ccaef-f225-415a-8353-43a49b96f06b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:40.409763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ccaef-f225-415a-8353-43a49b96f06b None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:40.410524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ccaef-f225-415a-8353-43a49b96f06b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:40.410867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ccaef-f225-415a-8353-43a49b96f06b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:40.412232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ccaef-f225-415a-8353-43a49b96f06b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:40.420534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 535/1063] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jun 29 08:23:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:40.426066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298964fa-b36f-4bc0-a693-de582c8e2895 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:40.533867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298964fa-b36f-4bc0-a693-de582c8e2895 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:40.568187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c134c7c-b77e-4c22-b497-ca3a3cb33c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:40.568974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134c7c-b77e-4c22-b497-ca3a3cb33c81 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:40.569250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134c7c-b77e-4c22-b497-ca3a3cb33c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:40.569527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c134c7c-b77e-4c22-b497-ca3a3cb33c81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:40.580641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c134c7c-b77e-4c22-b497-ca3a3cb33c81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:40.582081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c134c7c-b77e-4c22-b497-ca3a3cb33c81 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-2c134c7c-b77e-4c22-b497-ca3a3cb33c81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:40.599184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:40.600082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:40.600979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:40.600979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:40.600979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:40.600979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:40.603029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 529/1064] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:40.606333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ccaef-f225-415a-8353-43a49b96f06b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:40.708816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ccaef-f225-415a-8353-43a49b96f06b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:40.742556 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812998c9-8f5a-411c-b887-364bd57857fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:40.743253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812998c9-8f5a-411c-b887-364bd57857fa None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:40.743535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812998c9-8f5a-411c-b887-364bd57857fa None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:40.743801 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812998c9-8f5a-411c-b887-364bd57857fa None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:40.744554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812998c9-8f5a-411c-b887-364bd57857fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:40.744861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812998c9-8f5a-411c-b887-364bd57857fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:40.746228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812998c9-8f5a-411c-b887-364bd57857fa None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:40.759685 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 536/1065] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:40.765544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c134c7c-b77e-4c22-b497-ca3a3cb33c81 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:40.797812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:40.867648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c134c7c-b77e-4c22-b497-ca3a3cb33c81 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:40.900833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7297526-b193-417b-8c44-45800528f3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:40.901632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7297526-b193-417b-8c44-45800528f3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:40.901909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7297526-b193-417b-8c44-45800528f3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:40.902181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7297526-b193-417b-8c44-45800528f3b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:40.910127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7297526-b193-417b-8c44-45800528f3b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:40.911609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7297526-b193-417b-8c44-45800528f3b6 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d7297526-b193-417b-8c44-45800528f3b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:40.925632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:40.926559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:40.927554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:40.927554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:40.927554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:40.927554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:40.927554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:40.927554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:40.927554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:40.927554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:40.927554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:23:40.927554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:40.931032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 530/1066] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jun 29 08:23:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:40.932543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812998c9-8f5a-411c-b887-364bd57857fa None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:40.983390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:41.035408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812998c9-8f5a-411c-b887-364bd57857fa None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:41.070116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d39531-73d6-4345-b673-f5db986e89cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:41.070716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d39531-73d6-4345-b673-f5db986e89cb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.071014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d39531-73d6-4345-b673-f5db986e89cb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.071285 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d39531-73d6-4345-b673-f5db986e89cb None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.072120 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d39531-73d6-4345-b673-f5db986e89cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:41.072430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d39531-73d6-4345-b673-f5db986e89cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:41.073858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d39531-73d6-4345-b673-f5db986e89cb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:41.092202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 537/1067] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jun 29 08:23:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:41.098046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7297526-b193-417b-8c44-45800528f3b6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:41.201140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:41.211806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7297526-b193-417b-8c44-45800528f3b6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:41.212544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccbc16-3d5f-493b-9f96-e4d559f432fd None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.212835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccbc16-3d5f-493b-9f96-e4d559f432fd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.213108 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccbc16-3d5f-493b-9f96-e4d559f432fd None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.215098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 538/1068] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:41.219797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38040a20-5d28-495b-b4a5-97217484a6a9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.220345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38040a20-5d28-495b-b4a5-97217484a6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.220876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38040a20-5d28-495b-b4a5-97217484a6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.237357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-38040a20-5d28-495b-b4a5-97217484a6a9 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:41.238543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-38040a20-5d28-495b-b4a5-97217484a6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:41.255743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eee98e-8654-40c0-b6c8-d9d773d46cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:41.257243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eee98e-8654-40c0-b6c8-d9d773d46cbb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.257673 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eee98e-8654-40c0-b6c8-d9d773d46cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.258095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eee98e-8654-40c0-b6c8-d9d773d46cbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.273833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eee98e-8654-40c0-b6c8-d9d773d46cbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:41.276506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eee98e-8654-40c0-b6c8-d9d773d46cbb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-98eee98e-8654-40c0-b6c8-d9d773d46cbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:41.301611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:41.302586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:41.308707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:41.308707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:41.308707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:41.308707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:41.308707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 531/1069] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:41.310225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eee98e-8654-40c0-b6c8-d9d773d46cbb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:41.310988 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 539/1070] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:23:41.330623 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38040a20-5d28-495b-b4a5-97217484a6a9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:41.366492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:41.427442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eee98e-8654-40c0-b6c8-d9d773d46cbb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:41.433772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38040a20-5d28-495b-b4a5-97217484a6a9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:41.465537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d0409-3ba2-45ce-b33e-00b73d365647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:41.466215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d0409-3ba2-45ce-b33e-00b73d365647 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.466504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d0409-3ba2-45ce-b33e-00b73d365647 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.466831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d0409-3ba2-45ce-b33e-00b73d365647 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.467568 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d0409-3ba2-45ce-b33e-00b73d365647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:41.467873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d0409-3ba2-45ce-b33e-00b73d365647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:41.469250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d0409-3ba2-45ce-b33e-00b73d365647 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:41.476074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58e8d2-97e0-4164-9df9-d2b2bdc3e739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:41.477210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58e8d2-97e0-4164-9df9-d2b2bdc3e739 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.477466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58e8d2-97e0-4164-9df9-d2b2bdc3e739 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.477741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58e8d2-97e0-4164-9df9-d2b2bdc3e739 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.482478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 532/1071] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:41.487654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d0409-3ba2-45ce-b33e-00b73d365647 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:41.492060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:41.556252 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58e8d2-97e0-4164-9df9-d2b2bdc3e739 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:41.557720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58e8d2-97e0-4164-9df9-d2b2bdc3e739 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:41.592825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:41.593661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d0409-3ba2-45ce-b33e-00b73d365647 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:41.631489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb82c4-c169-4a89-aedf-0b404c39499e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:41.632606 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb82c4-c169-4a89-aedf-0b404c39499e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.633029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb82c4-c169-4a89-aedf-0b404c39499e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.633439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb82c4-c169-4a89-aedf-0b404c39499e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.640982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb82c4-c169-4a89-aedf-0b404c39499e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:41.642757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb82c4-c169-4a89-aedf-0b404c39499e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-9afb82c4-c169-4a89-aedf-0b404c39499e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:41.657487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:41.658486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:41.659539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:41.659539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:41.659539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:41.659539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:41.659539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:41.659539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:41.659539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:41.659539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:41.660268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 540/1072] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 08:23:41.662723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 533/1073] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:41.666019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58e8d2-97e0-4164-9df9-d2b2bdc3e739 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:41.667016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb82c4-c169-4a89-aedf-0b404c39499e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:41.728694 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:41.765686 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb82c4-c169-4a89-aedf-0b404c39499e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:41.766147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58e8d2-97e0-4164-9df9-d2b2bdc3e739 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:41.798670 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8901dd-1c16-4302-b8a3-2125df3d504a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:41.799497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8901dd-1c16-4302-b8a3-2125df3d504a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.799781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8901dd-1c16-4302-b8a3-2125df3d504a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.800053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8901dd-1c16-4302-b8a3-2125df3d504a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.800864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d6872-8eb8-485a-86b6-78e98f6f21ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:41.801520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d6872-8eb8-485a-86b6-78e98f6f21ba None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.801782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d6872-8eb8-485a-86b6-78e98f6f21ba None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.802066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d6872-8eb8-485a-86b6-78e98f6f21ba None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.802878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d6872-8eb8-485a-86b6-78e98f6f21ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:41.803015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:41.803186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d6872-8eb8-485a-86b6-78e98f6f21ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:41.804616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d6872-8eb8-485a-86b6-78e98f6f21ba None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:41.811256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8901dd-1c16-4302-b8a3-2125df3d504a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:41.812705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8901dd-1c16-4302-b8a3-2125df3d504a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:41.816530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 534/1074] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:41.817773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c642aea-2e0b-4a41-9993-dd6cec9df9cc None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.818054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c642aea-2e0b-4a41-9993-dd6cec9df9cc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.818326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c642aea-2e0b-4a41-9993-dd6cec9df9cc None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.819730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 535/1075] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:41.822500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c642aea-2e0b-4a41-9993-dd6cec9df9cc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ad8901dd-1c16-4302-b8a3-2125df3d504a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:41.833018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:41.834007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:41.835115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:41.835115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:41.835115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:41.835115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:41.835115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 541/1076] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:41.835570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee9bb-b78a-4af5-9ada-6bc3d9405fa0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.835766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee9bb-b78a-4af5-9ada-6bc3d9405fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.836031 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ee9bb-b78a-4af5-9ada-6bc3d9405fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.852327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-302ee9bb-b78a-4af5-9ada-6bc3d9405fa0 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:41.853432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-302ee9bb-b78a-4af5-9ada-6bc3d9405fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:41.921163 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 542/1077] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:41.921777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ee9bb-b78a-4af5-9ada-6bc3d9405fa0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:41.924816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c642aea-2e0b-4a41-9993-dd6cec9df9cc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:41.948986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:41.960993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842553ba-6a5f-4c6d-8e34-973094f95f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:41.961940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842553ba-6a5f-4c6d-8e34-973094f95f8a None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:41.962218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842553ba-6a5f-4c6d-8e34-973094f95f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:41.962487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842553ba-6a5f-4c6d-8e34-973094f95f8a None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/7bd500f020f143339c5a4fc9f0728ee4/roles/f21b1b6d7f944363b8107baa2cc893ca` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:41.965262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842553ba-6a5f-4c6d-8e34-973094f95f8a None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=7bd500f020f143339c5a4fc9f0728ee4, role_id=f21b1b6d7f944363b8107baa2cc893ca)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:41.966715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842553ba-6a5f-4c6d-8e34-973094f95f8a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:41.981871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 536/1078] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:23:41 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/7bd500f020f143339c5a4fc9f0728ee4/roles/f21b1b6d7f944363b8107baa2cc893ca => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:23:41.982661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842553ba-6a5f-4c6d-8e34-973094f95f8a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:42.031504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ee9bb-b78a-4af5-9ada-6bc3d9405fa0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:42.055677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:42.072037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484313fc-a883-4351-87a2-d07eff631c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:42.072723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484313fc-a883-4351-87a2-d07eff631c73 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:42.073007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484313fc-a883-4351-87a2-d07eff631c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:42.073283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484313fc-a883-4351-87a2-d07eff631c73 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:42.074028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484313fc-a883-4351-87a2-d07eff631c73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:42.074336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484313fc-a883-4351-87a2-d07eff631c73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:42.075797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484313fc-a883-4351-87a2-d07eff631c73 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:42.095016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 543/1079] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:42.103621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842553ba-6a5f-4c6d-8e34-973094f95f8a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:42.146007 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f454a6-5eab-435d-84ec-771ac6c32355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:42.147596 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f454a6-5eab-435d-84ec-771ac6c32355 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:42.147596 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f454a6-5eab-435d-84ec-771ac6c32355 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:42.147596 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f454a6-5eab-435d-84ec-771ac6c32355 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:42.160270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f454a6-5eab-435d-84ec-771ac6c32355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-58f454a6-5eab-435d-84ec-771ac6c32355 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:42.168115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:42.169149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:42.169976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 537/1080] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:23:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:42.177081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484313fc-a883-4351-87a2-d07eff631c73 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:42.291582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484313fc-a883-4351-87a2-d07eff631c73 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:42.327088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed35d35-0feb-457f-8ad3-269ce788a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:42.327767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed35d35-0feb-457f-8ad3-269ce788a68a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:42.328052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed35d35-0feb-457f-8ad3-269ce788a68a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:42.328328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed35d35-0feb-457f-8ad3-269ce788a68a None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:42.329085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed35d35-0feb-457f-8ad3-269ce788a68a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:42.329568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed35d35-0feb-457f-8ad3-269ce788a68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:42.330980 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed35d35-0feb-457f-8ad3-269ce788a68a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:42.340691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 544/1081] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:23:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:42.347093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f454a6-5eab-435d-84ec-771ac6c32355 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:42.397322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:42.452270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f454a6-5eab-435d-84ec-771ac6c32355 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:42.492757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577e6fb-04fe-4d7d-889b-4833344eb247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:42.493646 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577e6fb-04fe-4d7d-889b-4833344eb247 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:42.493930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577e6fb-04fe-4d7d-889b-4833344eb247 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:42.494210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577e6fb-04fe-4d7d-889b-4833344eb247 None admin] PATH_INFO: `/v3/users/2d8827247f0847178f2204329595ce17` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:42.495440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577e6fb-04fe-4d7d-889b-4833344eb247 None admin] RBAC: Authorizing `identity:get_user(user_id=2d8827247f0847178f2204329595ce17)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:42.496885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577e6fb-04fe-4d7d-889b-4833344eb247 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:42.498553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 538/1082] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:23:42 2022] GET /identity/v3/users/2d8827247f0847178f2204329595ce17 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:42.504553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed35d35-0feb-457f-8ad3-269ce788a68a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:42.549703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:42.608712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed35d35-0feb-457f-8ad3-269ce788a68a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:42.643135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2f14-343e-425e-86cc-63f26e22556e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:42.644008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2f14-343e-425e-86cc-63f26e22556e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:42.644284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2f14-343e-425e-86cc-63f26e22556e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:42.644557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2f14-343e-425e-86cc-63f26e22556e None admin] PATH_INFO: `/v3/projects/baa5396629c841728e8186c551bf073a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:42.645775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb2f14-343e-425e-86cc-63f26e22556e None admin] RBAC: Authorizing `identity:get_project(project_id=baa5396629c841728e8186c551bf073a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:42.647245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb2f14-343e-425e-86cc-63f26e22556e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:42.649010 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 545/1083] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:23:42 2022] GET /identity/v3/projects/baa5396629c841728e8186c551bf073a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:42.655022 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577e6fb-04fe-4d7d-889b-4833344eb247 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:42.760686 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577e6fb-04fe-4d7d-889b-4833344eb247 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:42.773649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:42.797227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00117b52-1c4d-4c2d-9344-6138c31c8830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:42.798295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00117b52-1c4d-4c2d-9344-6138c31c8830 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:42.798600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00117b52-1c4d-4c2d-9344-6138c31c8830 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:42.798910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00117b52-1c4d-4c2d-9344-6138c31c8830 None admin] PATH_INFO: `/v3/projects/baa5396629c841728e8186c551bf073a/users/2d8827247f0847178f2204329595ce17/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:42.800987 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00117b52-1c4d-4c2d-9344-6138c31c8830 None admin] RBAC: Authorizing `identity:create_grant(project_id=baa5396629c841728e8186c551bf073a, user_id=2d8827247f0847178f2204329595ce17, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:42.802424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00117b52-1c4d-4c2d-9344-6138c31c8830 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:42.818200 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 539/1084] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:23:42 2022] PUT /identity/v3/projects/baa5396629c841728e8186c551bf073a/users/2d8827247f0847178f2204329595ce17/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:23:44.025749 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9d703-487e-4a0d-8d94-be4a41b61747 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:44.026313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9d703-487e-4a0d-8d94-be4a41b61747 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:44.026313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9d703-487e-4a0d-8d94-be4a41b61747 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:44.027739 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 546/1085] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:44.033452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c42ce-31f6-41bb-9355-af52c575cd3e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:44.033878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c42ce-31f6-41bb-9355-af52c575cd3e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:44.034214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c42ce-31f6-41bb-9355-af52c575cd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:44.052543 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3d9c42ce-31f6-41bb-9355-af52c575cd3e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:44.054156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3d9c42ce-31f6-41bb-9355-af52c575cd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:44.123948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 540/1086] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:44.147658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9d703-487e-4a0d-8d94-be4a41b61747 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:44.254028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9d703-487e-4a0d-8d94-be4a41b61747 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:44.263609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:44.289623 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e496e-ac55-4c18-96bc-91258275a8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:44.290352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e496e-ac55-4c18-96bc-91258275a8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:44.290635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e496e-ac55-4c18-96bc-91258275a8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:44.290948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e496e-ac55-4c18-96bc-91258275a8d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:44.301291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e496e-ac55-4c18-96bc-91258275a8d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1d4e496e-ac55-4c18-96bc-91258275a8d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:44.309338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:44.310248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:44.311110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 547/1087] 10.222.0.27 () {56 vars in 1120 bytes} [Wed Jun 29 08:23:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:44.316855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c42ce-31f6-41bb-9355-af52c575cd3e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:44.421304 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c42ce-31f6-41bb-9355-af52c575cd3e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:44.422850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:44.464352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abb7ff-3efd-40fc-9a0a-077d30986a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:44.465027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abb7ff-3efd-40fc-9a0a-077d30986a4e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:44.465306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abb7ff-3efd-40fc-9a0a-077d30986a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:44.465575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abb7ff-3efd-40fc-9a0a-077d30986a4e None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:44.466319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abb7ff-3efd-40fc-9a0a-077d30986a4e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:44.466598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abb7ff-3efd-40fc-9a0a-077d30986a4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:44.468006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abb7ff-3efd-40fc-9a0a-077d30986a4e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:44.476231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 541/1088] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jun 29 08:23:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:44.481885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e496e-ac55-4c18-96bc-91258275a8d6 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:44.590863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e496e-ac55-4c18-96bc-91258275a8d6 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:44.628466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b8eba-00c7-49e0-87bd-264eba5a4c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:44.629248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b8eba-00c7-49e0-87bd-264eba5a4c04 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:44.629521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b8eba-00c7-49e0-87bd-264eba5a4c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:44.629789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b8eba-00c7-49e0-87bd-264eba5a4c04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:44.640777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b8eba-00c7-49e0-87bd-264eba5a4c04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:44.642210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b8eba-00c7-49e0-87bd-264eba5a4c04 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-545b8eba-00c7-49e0-87bd-264eba5a4c04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:44.659148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:44.660038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:44.660913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:44.660913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:44.660913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:44.660913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:44.660913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 548/1089] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:44.666239 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abb7ff-3efd-40fc-9a0a-077d30986a4e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:44.769051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abb7ff-3efd-40fc-9a0a-077d30986a4e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:44.803314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216079eb-b50b-48c7-b057-2710e2746ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:44.803973 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216079eb-b50b-48c7-b057-2710e2746ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:44.804257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216079eb-b50b-48c7-b057-2710e2746ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:44.804519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216079eb-b50b-48c7-b057-2710e2746ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:44.805244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216079eb-b50b-48c7-b057-2710e2746ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:44.805547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216079eb-b50b-48c7-b057-2710e2746ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:44.806929 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216079eb-b50b-48c7-b057-2710e2746ca1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:44.818864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 542/1090] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:44.824596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b8eba-00c7-49e0-87bd-264eba5a4c04 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:44.840029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68677c6c-a3bd-424f-b153-9b7c7ef04c83 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:44.840280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68677c6c-a3bd-424f-b153-9b7c7ef04c83 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:44.840575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68677c6c-a3bd-424f-b153-9b7c7ef04c83 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:44.841987 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 543/1091] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:23:44.846938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916d010-cee3-4569-b62d-15796fdee9e0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:44.847224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916d010-cee3-4569-b62d-15796fdee9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:44.847496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916d010-cee3-4569-b62d-15796fdee9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:44.849693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:44.864730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a916d010-cee3-4569-b62d-15796fdee9e0 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:44.865916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a916d010-cee3-4569-b62d-15796fdee9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:44.883026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:44.928422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b8eba-00c7-49e0-87bd-264eba5a4c04 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:44.932553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 544/1092] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:44.956577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916d010-cee3-4569-b62d-15796fdee9e0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:44.965456 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1fa80-bf1a-44b4-9039-e076b6873617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:44.966302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1fa80-bf1a-44b4-9039-e076b6873617 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:44.966554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1fa80-bf1a-44b4-9039-e076b6873617 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:44.966869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1fa80-bf1a-44b4-9039-e076b6873617 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:44.974925 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1fa80-bf1a-44b4-9039-e076b6873617 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:44.976360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1fa80-bf1a-44b4-9039-e076b6873617 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4fd1fa80-bf1a-44b4-9039-e076b6873617 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:23:44.991641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:44.992667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:44.993854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:44.993854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:44.993854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:23:44.993854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:23:44.993854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:23:44.993854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:23:44.993854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:23:44.993854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:23:44.993854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 08:23:44.993854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:44.993854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 549/1093] 10.222.0.27 () {56 vars in 1093 bytes} [Wed Jun 29 08:23:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:44.999863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1fa80-bf1a-44b4-9039-e076b6873617 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:45.055720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916d010-cee3-4569-b62d-15796fdee9e0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:45.083575 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:45.090240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69a1e6-b726-46c9-936a-c54bb84af0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:45.091091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69a1e6-b726-46c9-936a-c54bb84af0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:45.091379 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69a1e6-b726-46c9-936a-c54bb84af0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:45.091688 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69a1e6-b726-46c9-936a-c54bb84af0a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:45.101746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69a1e6-b726-46c9-936a-c54bb84af0a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:23:45.103985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1fa80-bf1a-44b4-9039-e076b6873617 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6e69a1e6-b726-46c9-936a-c54bb84af0a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:23:45.109109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:45.110078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:23:45.111241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 545/1094] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:23:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:45.116347 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69a1e6-b726-46c9-936a-c54bb84af0a6 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:45.132794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:45.139256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47844122-997a-4e88-bea9-77534ad4b7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:45.139927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844122-997a-4e88-bea9-77534ad4b7cc None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:45.140210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844122-997a-4e88-bea9-77534ad4b7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:45.140484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47844122-997a-4e88-bea9-77534ad4b7cc None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:45.141327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47844122-997a-4e88-bea9-77534ad4b7cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:45.141637 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47844122-997a-4e88-bea9-77534ad4b7cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:45.143541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47844122-997a-4e88-bea9-77534ad4b7cc None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:45.160831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 550/1095] 10.222.0.27 () {56 vars in 1133 bytes} [Wed Jun 29 08:23:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:45.166482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47844122-997a-4e88-bea9-77534ad4b7cc None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:45.222008 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69a1e6-b726-46c9-936a-c54bb84af0a6 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:45.256526 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785120a-566f-40a6-826c-3a8db78c6482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:45.257201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785120a-566f-40a6-826c-3a8db78c6482 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:45.257484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785120a-566f-40a6-826c-3a8db78c6482 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:45.257762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785120a-566f-40a6-826c-3a8db78c6482 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:45.258516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785120a-566f-40a6-826c-3a8db78c6482 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:45.258863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785120a-566f-40a6-826c-3a8db78c6482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:45.260262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785120a-566f-40a6-826c-3a8db78c6482 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:45.263236 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47844122-997a-4e88-bea9-77534ad4b7cc None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:45.268797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 546/1096] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:23:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:45.275425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785120a-566f-40a6-826c-3a8db78c6482 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:45.299696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81851ca9-578a-4fed-bfc3-f773491a68ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:45.300346 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81851ca9-578a-4fed-bfc3-f773491a68ad None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:45.300631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81851ca9-578a-4fed-bfc3-f773491a68ad None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:45.300904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81851ca9-578a-4fed-bfc3-f773491a68ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:45.313639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:45.360384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:45.382284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81851ca9-578a-4fed-bfc3-f773491a68ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:45.383799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81851ca9-578a-4fed-bfc3-f773491a68ad None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:45.384134 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785120a-566f-40a6-826c-3a8db78c6482 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:45.418466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc2ab9-4235-4bb5-bdfd-da50a088f864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:45.419380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc2ab9-4235-4bb5-bdfd-da50a088f864 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:45.419665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc2ab9-4235-4bb5-bdfd-da50a088f864 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:45.419942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc2ab9-4235-4bb5-bdfd-da50a088f864 None admin] PATH_INFO: `/v3/users/2d8827247f0847178f2204329595ce17` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:45.421141 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc2ab9-4235-4bb5-bdfd-da50a088f864 None admin] RBAC: Authorizing `identity:get_user(user_id=2d8827247f0847178f2204329595ce17)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:45.422576 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc2ab9-4235-4bb5-bdfd-da50a088f864 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:45.424813 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 547/1097] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:23:45 2022] GET /identity/v3/users/2d8827247f0847178f2204329595ce17 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:45.430657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc2ab9-4235-4bb5-bdfd-da50a088f864 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:45.487760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 551/1098] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 08:23:45.493450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81851ca9-578a-4fed-bfc3-f773491a68ad None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:45.531431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:45.534840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc2ab9-4235-4bb5-bdfd-da50a088f864 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:45.568610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8720-6f65-4862-8e99-ede569ef2e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:45.569260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8720-6f65-4862-8e99-ede569ef2e62 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:45.569541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8720-6f65-4862-8e99-ede569ef2e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:45.569812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f8720-6f65-4862-8e99-ede569ef2e62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:45.584759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:45.590292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81851ca9-578a-4fed-bfc3-f773491a68ad None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:45.627622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9486d76-b378-47f0-ac4b-d05a4956c355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:45.628349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486d76-b378-47f0-ac4b-d05a4956c355 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:45.628635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486d76-b378-47f0-ac4b-d05a4956c355 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:45.628909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9486d76-b378-47f0-ac4b-d05a4956c355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:45.639582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f8720-6f65-4862-8e99-ede569ef2e62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:45.640968 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9486d76-b378-47f0-ac4b-d05a4956c355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:45.641079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f8720-6f65-4862-8e99-ede569ef2e62 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:45.642579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9486d76-b378-47f0-ac4b-d05a4956c355 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-e9486d76-b378-47f0-ac4b-d05a4956c355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:23:45.662580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:45.663582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:45.664463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:23:45.664463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:23:45.664463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:23:45.664463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:45.666849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 552/1099] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:23:45.670213 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9486d76-b378-47f0-ac4b-d05a4956c355 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:45.760430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 548/1100] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 08:23:45.767260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8720-6f65-4862-8e99-ede569ef2e62 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:45.773221 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:45.784294 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9486d76-b378-47f0-ac4b-d05a4956c355 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:45.815042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:45.824568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f45654-899f-4487-a1ba-10128524f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:45.825310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f45654-899f-4487-a1ba-10128524f177 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:45.825656 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f45654-899f-4487-a1ba-10128524f177 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:45.825939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f45654-899f-4487-a1ba-10128524f177 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:45.826666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f45654-899f-4487-a1ba-10128524f177 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:45.827005 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f45654-899f-4487-a1ba-10128524f177 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:45.828384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f45654-899f-4487-a1ba-10128524f177 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:45.840163 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 553/1101] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:45.845752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f45654-899f-4487-a1ba-10128524f177 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:45.871579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f8720-6f65-4862-8e99-ede569ef2e62 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:45.908760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d311a16-7818-42f2-837c-039ff05c8468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:45.909554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d311a16-7818-42f2-837c-039ff05c8468 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:45.909843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d311a16-7818-42f2-837c-039ff05c8468 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:45.910124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d311a16-7818-42f2-837c-039ff05c8468 None admin] PATH_INFO: `/v3/projects/baa5396629c841728e8186c551bf073a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:45.911341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d311a16-7818-42f2-837c-039ff05c8468 None admin] RBAC: Authorizing `identity:get_project(project_id=baa5396629c841728e8186c551bf073a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:45.912788 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d311a16-7818-42f2-837c-039ff05c8468 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:45.914842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 549/1102] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:23:45 2022] GET /identity/v3/projects/baa5396629c841728e8186c551bf073a => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:45.920549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d311a16-7818-42f2-837c-039ff05c8468 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:45.958322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f45654-899f-4487-a1ba-10128524f177 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:45.993465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4769e3-1e73-45cb-aa32-f615297c98d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:45.994267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4769e3-1e73-45cb-aa32-f615297c98d9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:45.994555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4769e3-1e73-45cb-aa32-f615297c98d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:45.994868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4769e3-1e73-45cb-aa32-f615297c98d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:46.001662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4769e3-1e73-45cb-aa32-f615297c98d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:46.003257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4769e3-1e73-45cb-aa32-f615297c98d9 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1e4769e3-1e73-45cb-aa32-f615297c98d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:23:46.017961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:23:46.019099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:23:46.019989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:23:46.019989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:23:46.019989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:23:46.019989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:23:46.019989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:23:46.019989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:23:46.019989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:23:46.019989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:23:46.019989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 554/1103] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:23:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:23:46.023532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d311a16-7818-42f2-837c-039ff05c8468 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:46.024966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4769e3-1e73-45cb-aa32-f615297c98d9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:46.036953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:46.038262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:46.059611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eed56a-3715-4b99-ac9a-bf2f6ae2e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:46.060276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eed56a-3715-4b99-ac9a-bf2f6ae2e40b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:46.060558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eed56a-3715-4b99-ac9a-bf2f6ae2e40b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:46.060836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eed56a-3715-4b99-ac9a-bf2f6ae2e40b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:46.061677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eed56a-3715-4b99-ac9a-bf2f6ae2e40b None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=baa5396629c841728e8186c551bf073a, user.id=2d8827247f0847178f2204329595ce17) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:46.061989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eed56a-3715-4b99-ac9a-bf2f6ae2e40b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:46.063456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eed56a-3715-4b99-ac9a-bf2f6ae2e40b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:46.070011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 550/1104] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:45 2022] GET /identity/v3/role_assignments?user.id=2d8827247f0847178f2204329595ce17&scope.project.id=baa5396629c841728e8186c551bf073a&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:46.133757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4769e3-1e73-45cb-aa32-f615297c98d9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:46.172300 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea27c2-230b-4cef-a364-361a5497d151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:46.172916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea27c2-230b-4cef-a364-361a5497d151 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:46.173194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea27c2-230b-4cef-a364-361a5497d151 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:46.174148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea27c2-230b-4cef-a364-361a5497d151 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:46.175007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea27c2-230b-4cef-a364-361a5497d151 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:46.175328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea27c2-230b-4cef-a364-361a5497d151 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:46.176905 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea27c2-230b-4cef-a364-361a5497d151 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:46.195667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 555/1105] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:23:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:46.196998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eed56a-3715-4b99-ac9a-bf2f6ae2e40b None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:46.300750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eed56a-3715-4b99-ac9a-bf2f6ae2e40b None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:46.333747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96901c98-c253-48d9-bc92-68e4a4b812c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:46.334408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96901c98-c253-48d9-bc92-68e4a4b812c4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:46.334706 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96901c98-c253-48d9-bc92-68e4a4b812c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:46.334982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96901c98-c253-48d9-bc92-68e4a4b812c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:46.335819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96901c98-c253-48d9-bc92-68e4a4b812c4 None admin] RBAC: Adding query filter params (role.id=f21b1b6d7f944363b8107baa2cc893ca, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=7bd500f020f143339c5a4fc9f0728ee4) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:46.336131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96901c98-c253-48d9-bc92-68e4a4b812c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:46.337561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96901c98-c253-48d9-bc92-68e4a4b812c4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:46.343904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 551/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:46 2022] GET /identity/v3/role_assignments?user.id=7bd500f020f143339c5a4fc9f0728ee4&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=f21b1b6d7f944363b8107baa2cc893ca => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:48.137756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2dc3-d2c9-4c57-b89e-ebaca96ed037 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:48.141037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2dc3-d2c9-4c57-b89e-ebaca96ed037 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:48.141037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2dc3-d2c9-4c57-b89e-ebaca96ed037 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:48.141037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 556/1107] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:48.146459 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77023b3-f4c2-41bd-838c-054d7e7d5982 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:48.146942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77023b3-f4c2-41bd-838c-054d7e7d5982 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:48.147044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77023b3-f4c2-41bd-838c-054d7e7d5982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:48.166425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d77023b3-f4c2-41bd-838c-054d7e7d5982 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:48.167668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d77023b3-f4c2-41bd-838c-054d7e7d5982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:48.218925 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:48.253372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 552/1108] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:48.278594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d2dc3-d2c9-4c57-b89e-ebaca96ed037 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:48.354050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:48.384369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d2dc3-d2c9-4c57-b89e-ebaca96ed037 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:48.419073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56a6a2-1363-4d1e-ac4e-805e921c6341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:48.419892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56a6a2-1363-4d1e-ac4e-805e921c6341 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:48.420170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56a6a2-1363-4d1e-ac4e-805e921c6341 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:48.420445 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56a6a2-1363-4d1e-ac4e-805e921c6341 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:48.421765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56a6a2-1363-4d1e-ac4e-805e921c6341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:48.423247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56a6a2-1363-4d1e-ac4e-805e921c6341 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:48.425179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 557/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:48.431493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77023b3-f4c2-41bd-838c-054d7e7d5982 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:48.537114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77023b3-f4c2-41bd-838c-054d7e7d5982 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:48.572933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4e9ff-6da3-409d-a1da-1ce2e9d073d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:48.573608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e9ff-6da3-409d-a1da-1ce2e9d073d8 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:48.573893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e9ff-6da3-409d-a1da-1ce2e9d073d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:48.574177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4e9ff-6da3-409d-a1da-1ce2e9d073d8 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:48.574923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4e9ff-6da3-409d-a1da-1ce2e9d073d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:48.576373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b4e9ff-6da3-409d-a1da-1ce2e9d073d8 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:48.601011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 553/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:23:48 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:23:50.668380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3320f2-11d6-48f0-b516-3b0980e6d2cd None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:50.669271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3320f2-11d6-48f0-b516-3b0980e6d2cd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:50.669271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3320f2-11d6-48f0-b516-3b0980e6d2cd None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:50.671576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 558/1111] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:50.677092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6bc53-f7c5-43f3-a365-6d178048ebcd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:50.677520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6bc53-f7c5-43f3-a365-6d178048ebcd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:50.677649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6bc53-f7c5-43f3-a365-6d178048ebcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:50.695897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-03b6bc53-f7c5-43f3-a365-6d178048ebcd None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:50.697070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-03b6bc53-f7c5-43f3-a365-6d178048ebcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:50.717755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:50.766149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 554/1112] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:23:50.797794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3320f2-11d6-48f0-b516-3b0980e6d2cd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:50.913459 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3320f2-11d6-48f0-b516-3b0980e6d2cd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:50.959850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356e84d-eb6e-46c1-8450-060e7586a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:50.960701 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356e84d-eb6e-46c1-8450-060e7586a48a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:50.960983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356e84d-eb6e-46c1-8450-060e7586a48a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:50.961221 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356e84d-eb6e-46c1-8450-060e7586a48a None admin] PATH_INFO: `/v3/roles/3531cd7306964cc5967f92c559890ae5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:50.962880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356e84d-eb6e-46c1-8450-060e7586a48a None admin] RBAC: Authorizing `identity:get_role(role_id=3531cd7306964cc5967f92c559890ae5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:50.964615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356e84d-eb6e-46c1-8450-060e7586a48a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:50.965984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 559/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:23:50 2022] GET /identity/v3/roles/3531cd7306964cc5967f92c559890ae5 => generated 223 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:50.972839 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6bc53-f7c5-43f3-a365-6d178048ebcd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:51.083919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6bc53-f7c5-43f3-a365-6d178048ebcd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:51.120215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7164052-2145-4718-918d-8091e0121b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:51.121084 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7164052-2145-4718-918d-8091e0121b40 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:51.121364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7164052-2145-4718-918d-8091e0121b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:51.121631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7164052-2145-4718-918d-8091e0121b40 None admin] PATH_INFO: `/v3/users/88cabbc8e68e4954982c4ef0406b2124` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:51.135257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7164052-2145-4718-918d-8091e0121b40 None admin] RBAC: Authorizing `identity:get_user(user_id=88cabbc8e68e4954982c4ef0406b2124)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:51.136688 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7164052-2145-4718-918d-8091e0121b40 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:51.138344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 555/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:23:50 2022] GET /identity/v3/users/88cabbc8e68e4954982c4ef0406b2124 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:51.144691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356e84d-eb6e-46c1-8450-060e7586a48a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:51.154120 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:51.252257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356e84d-eb6e-46c1-8450-060e7586a48a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:51.287039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c17e6-9273-4ec0-aacc-2481b541afa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:51.287685 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c17e6-9273-4ec0-aacc-2481b541afa0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:51.287964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c17e6-9273-4ec0-aacc-2481b541afa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:51.288239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c17e6-9273-4ec0-aacc-2481b541afa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:51.358307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c17e6-9273-4ec0-aacc-2481b541afa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:51.359782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c17e6-9273-4ec0-aacc-2481b541afa0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:51.367454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:51.461432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 560/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:23:51.468029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7164052-2145-4718-918d-8091e0121b40 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:51.485175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:51.570249 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7164052-2145-4718-918d-8091e0121b40 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:51.606070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7509d66-0a23-4546-b2c6-b30068cd9ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:51.606950 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7509d66-0a23-4546-b2c6-b30068cd9ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:51.607228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7509d66-0a23-4546-b2c6-b30068cd9ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:51.607499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7509d66-0a23-4546-b2c6-b30068cd9ec4 None admin] PATH_INFO: `/v3/projects/baa5396629c841728e8186c551bf073a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:51.608721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7509d66-0a23-4546-b2c6-b30068cd9ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=baa5396629c841728e8186c551bf073a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:51.610153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7509d66-0a23-4546-b2c6-b30068cd9ec4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:51.611873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 556/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:23:51 2022] GET /identity/v3/projects/baa5396629c841728e8186c551bf073a => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:51.618755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c17e6-9273-4ec0-aacc-2481b541afa0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:51.728952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c17e6-9273-4ec0-aacc-2481b541afa0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:51.736234 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:51.765482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7591e1-9b1d-4dbd-9c39-861f4703b0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:51.766126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7591e1-9b1d-4dbd-9c39-861f4703b0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:51.766406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7591e1-9b1d-4dbd-9c39-861f4703b0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:51.766680 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7591e1-9b1d-4dbd-9c39-861f4703b0f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:51.767549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7591e1-9b1d-4dbd-9c39-861f4703b0f6 None admin] RBAC: Adding query filter params (role.id=3531cd7306964cc5967f92c559890ae5, scope.project.id=baa5396629c841728e8186c551bf073a, user.id=88cabbc8e68e4954982c4ef0406b2124) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:51.767862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7591e1-9b1d-4dbd-9c39-861f4703b0f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:51.769412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7591e1-9b1d-4dbd-9c39-861f4703b0f6 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:51.775054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 561/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:51 2022] GET /identity/v3/role_assignments?user.id=88cabbc8e68e4954982c4ef0406b2124&scope.project.id=baa5396629c841728e8186c551bf073a&role.id=3531cd7306964cc5967f92c559890ae5 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:53.755517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acecf63-f7fa-4037-8916-9d715d7c3b4f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:53.756096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acecf63-f7fa-4037-8916-9d715d7c3b4f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:53.756096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acecf63-f7fa-4037-8916-9d715d7c3b4f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:53.757381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 557/1118] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:53.763887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9fb5a-fee9-4642-a89e-6b85a77947a4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:53.764645 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9fb5a-fee9-4642-a89e-6b85a77947a4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:53.764765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9fb5a-fee9-4642-a89e-6b85a77947a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:53.790385 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4cc9fb5a-fee9-4642-a89e-6b85a77947a4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:53.791621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4cc9fb5a-fee9-4642-a89e-6b85a77947a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:53.862389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 562/1119] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:53.886372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acecf63-f7fa-4037-8916-9d715d7c3b4f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:53.977933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:53.999111 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acecf63-f7fa-4037-8916-9d715d7c3b4f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:54.035931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0d5be-895a-4f01-ac0a-541baf8045e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:54.036721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0d5be-895a-4f01-ac0a-541baf8045e0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:54.037003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0d5be-895a-4f01-ac0a-541baf8045e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:54.037275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0d5be-895a-4f01-ac0a-541baf8045e0 None admin] PATH_INFO: `/v3/roles/3531cd7306964cc5967f92c559890ae5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:54.038831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0d5be-895a-4f01-ac0a-541baf8045e0 None admin] RBAC: Authorizing `identity:get_role(role_id=3531cd7306964cc5967f92c559890ae5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:54.040217 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0d5be-895a-4f01-ac0a-541baf8045e0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:54.041592 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 558/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:23:53 2022] GET /identity/v3/roles/3531cd7306964cc5967f92c559890ae5 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:54.048169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9fb5a-fee9-4642-a89e-6b85a77947a4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:54.154073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:54.158948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9fb5a-fee9-4642-a89e-6b85a77947a4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:54.201941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12c73e-2a4e-4216-8f73-3b2002a917ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:54.202861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c73e-2a4e-4216-8f73-3b2002a917ac None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:54.203145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c73e-2a4e-4216-8f73-3b2002a917ac None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:54.203423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12c73e-2a4e-4216-8f73-3b2002a917ac None admin] PATH_INFO: `/v3/users/88cabbc8e68e4954982c4ef0406b2124` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:54.204647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12c73e-2a4e-4216-8f73-3b2002a917ac None admin] RBAC: Authorizing `identity:get_user(user_id=88cabbc8e68e4954982c4ef0406b2124)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:54.206214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12c73e-2a4e-4216-8f73-3b2002a917ac None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:54.208046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 563/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:23:54 2022] GET /identity/v3/users/88cabbc8e68e4954982c4ef0406b2124 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:54.217471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0d5be-895a-4f01-ac0a-541baf8045e0 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:54.330420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0d5be-895a-4f01-ac0a-541baf8045e0 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:54.366231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d808c-7056-4e51-a4ff-ed301eb8ced0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:54.367126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d808c-7056-4e51-a4ff-ed301eb8ced0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:54.367406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d808c-7056-4e51-a4ff-ed301eb8ced0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:54.367682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d808c-7056-4e51-a4ff-ed301eb8ced0 None admin] PATH_INFO: `/v3/projects/baa5396629c841728e8186c551bf073a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:54.368992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d808c-7056-4e51-a4ff-ed301eb8ced0 None admin] RBAC: Authorizing `identity:get_project(project_id=baa5396629c841728e8186c551bf073a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:54.370431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d808c-7056-4e51-a4ff-ed301eb8ced0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:54.372190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 559/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:23:54 2022] GET /identity/v3/projects/baa5396629c841728e8186c551bf073a => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:23:54.381600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12c73e-2a4e-4216-8f73-3b2002a917ac None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:54.514521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12c73e-2a4e-4216-8f73-3b2002a917ac None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:54.550908 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b094e-4e53-4856-9c14-c0b9730fdc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:54.551818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b094e-4e53-4856-9c14-c0b9730fdc01 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:54.552101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b094e-4e53-4856-9c14-c0b9730fdc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:54.552379 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b094e-4e53-4856-9c14-c0b9730fdc01 None admin] PATH_INFO: `/v3/projects/baa5396629c841728e8186c551bf073a/users/88cabbc8e68e4954982c4ef0406b2124/roles/3531cd7306964cc5967f92c559890ae5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:54.555302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b094e-4e53-4856-9c14-c0b9730fdc01 None admin] RBAC: Authorizing `identity:create_grant(project_id=baa5396629c841728e8186c551bf073a, user_id=88cabbc8e68e4954982c4ef0406b2124, role_id=3531cd7306964cc5967f92c559890ae5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:54.556786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b094e-4e53-4856-9c14-c0b9730fdc01 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:54.576859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 564/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:23:54 2022] PUT /identity/v3/projects/baa5396629c841728e8186c551bf073a/users/88cabbc8e68e4954982c4ef0406b2124/roles/3531cd7306964cc5967f92c559890ae5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:23:56.555868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967b9d1-fd05-476b-be9b-96b9781c9181 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:56.556744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967b9d1-fd05-476b-be9b-96b9781c9181 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:56.556744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967b9d1-fd05-476b-be9b-96b9781c9181 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:56.558969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 560/1124] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:56.567126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbac14-a5ac-47e7-8920-025319657cde None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:56.567536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbac14-a5ac-47e7-8920-025319657cde None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:56.567675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbac14-a5ac-47e7-8920-025319657cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:56.585958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5bfbac14-a5ac-47e7-8920-025319657cde None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:56.587183 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5bfbac14-a5ac-47e7-8920-025319657cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:56.613325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:56.661396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 565/1125] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:56.685139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967b9d1-fd05-476b-be9b-96b9781c9181 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:56.710271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:56.810528 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967b9d1-fd05-476b-be9b-96b9781c9181 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:56.845311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac63d99-167f-448d-a8ee-ed867f26d625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:56.846157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac63d99-167f-448d-a8ee-ed867f26d625 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:56.846502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac63d99-167f-448d-a8ee-ed867f26d625 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:56.846776 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac63d99-167f-448d-a8ee-ed867f26d625 None admin] PATH_INFO: `/v3/roles/3531cd7306964cc5967f92c559890ae5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:56.848301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac63d99-167f-448d-a8ee-ed867f26d625 None admin] RBAC: Authorizing `identity:get_role(role_id=3531cd7306964cc5967f92c559890ae5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:56.849669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac63d99-167f-448d-a8ee-ed867f26d625 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:56.851069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 561/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:23:56 2022] GET /identity/v3/roles/3531cd7306964cc5967f92c559890ae5 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:56.857029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbac14-a5ac-47e7-8920-025319657cde None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:56.962500 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbac14-a5ac-47e7-8920-025319657cde None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:56.997505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21b16c-9753-4003-931d-93c22bdd81e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:56.998358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21b16c-9753-4003-931d-93c22bdd81e0 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:56.998635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21b16c-9753-4003-931d-93c22bdd81e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:56.998938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21b16c-9753-4003-931d-93c22bdd81e0 None admin] PATH_INFO: `/v3/users/88cabbc8e68e4954982c4ef0406b2124` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:57.000225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21b16c-9753-4003-931d-93c22bdd81e0 None admin] RBAC: Authorizing `identity:get_user(user_id=88cabbc8e68e4954982c4ef0406b2124)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:57.001651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21b16c-9753-4003-931d-93c22bdd81e0 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:57.003303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 566/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:23:56 2022] GET /identity/v3/users/88cabbc8e68e4954982c4ef0406b2124 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:57.009577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac63d99-167f-448d-a8ee-ed867f26d625 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:57.107059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:57.123609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac63d99-167f-448d-a8ee-ed867f26d625 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:57.158816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa66c2-fe76-434a-94b1-5cb42a9660d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:57.159704 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa66c2-fe76-434a-94b1-5cb42a9660d1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:57.160172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa66c2-fe76-434a-94b1-5cb42a9660d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:57.160420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa66c2-fe76-434a-94b1-5cb42a9660d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:57.230135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa66c2-fe76-434a-94b1-5cb42a9660d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:57.231562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa66c2-fe76-434a-94b1-5cb42a9660d1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:57.315869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:57.332544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 562/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:23:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:23:57.339238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21b16c-9753-4003-931d-93c22bdd81e0 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:57.371774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:57.450062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21b16c-9753-4003-931d-93c22bdd81e0 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:57.487650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693a9d9-3d59-42dc-8a90-704413f53338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:57.488458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693a9d9-3d59-42dc-8a90-704413f53338 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:57.488738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693a9d9-3d59-42dc-8a90-704413f53338 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:57.489012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693a9d9-3d59-42dc-8a90-704413f53338 None admin] PATH_INFO: `/v3/projects/baa5396629c841728e8186c551bf073a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:57.490318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693a9d9-3d59-42dc-8a90-704413f53338 None admin] RBAC: Authorizing `identity:get_project(project_id=baa5396629c841728e8186c551bf073a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:57.491784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693a9d9-3d59-42dc-8a90-704413f53338 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:57.493490 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 567/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:23:57 2022] GET /identity/v3/projects/baa5396629c841728e8186c551bf073a => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:57.499983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa66c2-fe76-434a-94b1-5cb42a9660d1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:57.605262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa66c2-fe76-434a-94b1-5cb42a9660d1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:57.640435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62d488-10b2-4be6-869b-3eb6a5c9331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:57.641072 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62d488-10b2-4be6-869b-3eb6a5c9331c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:57.641348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62d488-10b2-4be6-869b-3eb6a5c9331c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:57.641618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62d488-10b2-4be6-869b-3eb6a5c9331c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:57.642446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62d488-10b2-4be6-869b-3eb6a5c9331c None admin] RBAC: Adding query filter params (role.id=3531cd7306964cc5967f92c559890ae5, scope.project.id=baa5396629c841728e8186c551bf073a, user.id=88cabbc8e68e4954982c4ef0406b2124) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:23:57.642783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62d488-10b2-4be6-869b-3eb6a5c9331c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:57.644490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62d488-10b2-4be6-869b-3eb6a5c9331c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:57.650827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 563/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:23:57 2022] GET /identity/v3/role_assignments?user.id=88cabbc8e68e4954982c4ef0406b2124&scope.project.id=baa5396629c841728e8186c551bf073a&role.id=3531cd7306964cc5967f92c559890ae5 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:59.674601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f03509-2f50-421e-87b4-37ecf0fc9ab9 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:59.675611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f03509-2f50-421e-87b4-37ecf0fc9ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:59.675611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f03509-2f50-421e-87b4-37ecf0fc9ab9 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:59.677867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 568/1131] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:23:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:23:59.684165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b628657-3a95-4aee-aaf0-5e61707ecf4d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:59.684617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b628657-3a95-4aee-aaf0-5e61707ecf4d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:59.684715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b628657-3a95-4aee-aaf0-5e61707ecf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:59.704745 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5b628657-3a95-4aee-aaf0-5e61707ecf4d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:23:59.705930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5b628657-3a95-4aee-aaf0-5e61707ecf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:23:59.760359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:59.787238 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 564/1132] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:23:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:23:59.811546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f03509-2f50-421e-87b4-37ecf0fc9ab9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:23:59.929645 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:23:59.938181 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f03509-2f50-421e-87b4-37ecf0fc9ab9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:23:59.974098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb67f359-cbf4-41e5-aaf1-c85911ec0def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:23:59.974916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67f359-cbf4-41e5-aaf1-c85911ec0def None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:23:59.975198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67f359-cbf4-41e5-aaf1-c85911ec0def None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:23:59.975470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67f359-cbf4-41e5-aaf1-c85911ec0def None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:23:59.976767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67f359-cbf4-41e5-aaf1-c85911ec0def None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:23:59.978195 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb67f359-cbf4-41e5-aaf1-c85911ec0def None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:23:59.979834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 569/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:23:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:23:59.986571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b628657-3a95-4aee-aaf0-5e61707ecf4d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:00.094509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b628657-3a95-4aee-aaf0-5e61707ecf4d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:00.129344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac1c7b-10fa-41a7-b87b-257acaf5d8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:00.129988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac1c7b-10fa-41a7-b87b-257acaf5d8a7 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:00.130268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac1c7b-10fa-41a7-b87b-257acaf5d8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:00.130539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac1c7b-10fa-41a7-b87b-257acaf5d8a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:00.131267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac1c7b-10fa-41a7-b87b-257acaf5d8a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:00.132687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac1c7b-10fa-41a7-b87b-257acaf5d8a7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:00.162473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 565/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:23:59 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:24:02.209705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a7133-be08-406d-89c4-1a1754a04304 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:02.210604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a7133-be08-406d-89c4-1a1754a04304 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:02.210604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a7133-be08-406d-89c4-1a1754a04304 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:02.212875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 570/1135] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:24:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:24:02.218782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264892a-c797-43a3-88ac-aeb1e099b55a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:02.219212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264892a-c797-43a3-88ac-aeb1e099b55a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:02.219302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264892a-c797-43a3-88ac-aeb1e099b55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:02.238069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6264892a-c797-43a3-88ac-aeb1e099b55a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:24:02.239294 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6264892a-c797-43a3-88ac-aeb1e099b55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:24:02.265585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:02.319323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 566/1136] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:24:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:24:02.350590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a7133-be08-406d-89c4-1a1754a04304 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:02.460428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a7133-be08-406d-89c4-1a1754a04304 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:02.499081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea998c-8e55-4342-b8bf-bd91440c315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:02.499872 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea998c-8e55-4342-b8bf-bd91440c315a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:02.500160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea998c-8e55-4342-b8bf-bd91440c315a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:02.500435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea998c-8e55-4342-b8bf-bd91440c315a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:02.501652 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea998c-8e55-4342-b8bf-bd91440c315a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:02.503216 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ea998c-8e55-4342-b8bf-bd91440c315a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:02.504802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 571/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:24:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:02.510876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264892a-c797-43a3-88ac-aeb1e099b55a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:02.617598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264892a-c797-43a3-88ac-aeb1e099b55a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:02.656458 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e610ab-636b-4be5-b81c-727cbe090f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:02.657132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e610ab-636b-4be5-b81c-727cbe090f15 None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:02.657420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e610ab-636b-4be5-b81c-727cbe090f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:02.657703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e610ab-636b-4be5-b81c-727cbe090f15 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:02.658419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e610ab-636b-4be5-b81c-727cbe090f15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:02.660040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e610ab-636b-4be5-b81c-727cbe090f15 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:02.684105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 567/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:24:02 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:24:04.767308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6c635-e1b0-4a20-aaa5-ab23baf05746 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:04.767877 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6c635-e1b0-4a20-aaa5-ab23baf05746 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:04.767877 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6c635-e1b0-4a20-aaa5-ab23baf05746 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:04.769289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 572/1139] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:24:04.775729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0c14a-a6b9-414a-a50b-253840df838b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:04.776427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0c14a-a6b9-414a-a50b-253840df838b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:04.776654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0c14a-a6b9-414a-a50b-253840df838b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:04.804059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c8b0c14a-a6b9-414a-a50b-253840df838b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:24:04.805270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c8b0c14a-a6b9-414a-a50b-253840df838b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:24:04.821610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:04.889075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 568/1140] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:24:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:24:04.922388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6c635-e1b0-4a20-aaa5-ab23baf05746 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:04.961761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:05.039424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6c635-e1b0-4a20-aaa5-ab23baf05746 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:05.075849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaef913-c5ed-4baa-ba84-bd48fea7a0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:05.076636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaef913-c5ed-4baa-ba84-bd48fea7a0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:05.076919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaef913-c5ed-4baa-ba84-bd48fea7a0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:05.077178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaef913-c5ed-4baa-ba84-bd48fea7a0a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:05.087548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaef913-c5ed-4baa-ba84-bd48fea7a0a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5aaef913-c5ed-4baa-ba84-bd48fea7a0a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:24:05.095439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:24:05.096489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:24:05.097686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 573/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:24:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:24:05.104996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0c14a-a6b9-414a-a50b-253840df838b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:05.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0c14a-a6b9-414a-a50b-253840df838b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:05.258549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d054a8c-1122-48d1-b6e5-c607bba1b6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:05.259631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d054a8c-1122-48d1-b6e5-c607bba1b6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:05.260069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d054a8c-1122-48d1-b6e5-c607bba1b6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:05.260506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d054a8c-1122-48d1-b6e5-c607bba1b6b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:05.261717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d054a8c-1122-48d1-b6e5-c607bba1b6b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:24:05.262187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d054a8c-1122-48d1-b6e5-c607bba1b6b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:05.264462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d054a8c-1122-48d1-b6e5-c607bba1b6b1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:05.274622 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 569/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:24:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:24:05.280489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaef913-c5ed-4baa-ba84-bd48fea7a0a1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:05.383298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:05.398364 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaef913-c5ed-4baa-ba84-bd48fea7a0a1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:05.435685 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d7721-a0f4-46dd-86e2-a0ad490c98ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:05.436544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d7721-a0f4-46dd-86e2-a0ad490c98ce None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:05.436829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d7721-a0f4-46dd-86e2-a0ad490c98ce None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:05.437096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d7721-a0f4-46dd-86e2-a0ad490c98ce None admin] PATH_INFO: `/v3/users/4de00d64c8e94a7982cc34a9f6b9b9d9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:05.452068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d7721-a0f4-46dd-86e2-a0ad490c98ce None admin] RBAC: Authorizing `identity:get_user(user_id=4de00d64c8e94a7982cc34a9f6b9b9d9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:05.453549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d7721-a0f4-46dd-86e2-a0ad490c98ce None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:05.459085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 574/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:24:05 2022] GET /identity/v3/users/4de00d64c8e94a7982cc34a9f6b9b9d9 => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:05.462018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d054a8c-1122-48d1-b6e5-c607bba1b6b1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:05.482919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:05.579706 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d054a8c-1122-48d1-b6e5-c607bba1b6b1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:05.620069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693dc611-1628-4918-97b3-76fcaa146cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:05.620711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693dc611-1628-4918-97b3-76fcaa146cef None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:05.620997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693dc611-1628-4918-97b3-76fcaa146cef None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:05.621268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693dc611-1628-4918-97b3-76fcaa146cef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:05.690494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693dc611-1628-4918-97b3-76fcaa146cef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:05.691928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693dc611-1628-4918-97b3-76fcaa146cef None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:05.704088 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:05.798972 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 570/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:24:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:24:05.807304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d7721-a0f4-46dd-86e2-a0ad490c98ce None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:05.923385 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d7721-a0f4-46dd-86e2-a0ad490c98ce None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:05.959257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb988059-6a89-4dd0-a5ce-020dee3f8f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:05.960040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb988059-6a89-4dd0-a5ce-020dee3f8f6b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:05.960323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb988059-6a89-4dd0-a5ce-020dee3f8f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:05.960601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb988059-6a89-4dd0-a5ce-020dee3f8f6b None admin] PATH_INFO: `/v3/projects/dc8789a78400438bb213ce4017660495` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:05.961847 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb988059-6a89-4dd0-a5ce-020dee3f8f6b None admin] RBAC: Authorizing `identity:get_project(project_id=dc8789a78400438bb213ce4017660495)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:05.963337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb988059-6a89-4dd0-a5ce-020dee3f8f6b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:05.965385 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 575/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:24:05 2022] GET /identity/v3/projects/dc8789a78400438bb213ce4017660495 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:05.971922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693dc611-1628-4918-97b3-76fcaa146cef None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:06.075565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693dc611-1628-4918-97b3-76fcaa146cef None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:06.086270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:06.112749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659e8e6-ce4f-4f34-a86f-54d3ac5a78a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:06.113466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e8e6-ce4f-4f34-a86f-54d3ac5a78a4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:06.113752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e8e6-ce4f-4f34-a86f-54d3ac5a78a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:06.114035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659e8e6-ce4f-4f34-a86f-54d3ac5a78a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:06.114914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659e8e6-ce4f-4f34-a86f-54d3ac5a78a4 None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=dc8789a78400438bb213ce4017660495, user.id=4de00d64c8e94a7982cc34a9f6b9b9d9) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:24:06.115230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659e8e6-ce4f-4f34-a86f-54d3ac5a78a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:06.116687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659e8e6-ce4f-4f34-a86f-54d3ac5a78a4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:06.122374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 571/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:24:05 2022] GET /identity/v3/role_assignments?user.id=4de00d64c8e94a7982cc34a9f6b9b9d9&scope.project.id=dc8789a78400438bb213ce4017660495&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:08.102599 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60b3f6-f875-4666-a1b2-5fc26f3b74ea None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:08.103558 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60b3f6-f875-4666-a1b2-5fc26f3b74ea None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:08.103733 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60b3f6-f875-4666-a1b2-5fc26f3b74ea None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:08.106008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 576/1147] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:24:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:24:08.111519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36328ca-4407-4b42-a266-8b20caf564dc None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:08.111955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36328ca-4407-4b42-a266-8b20caf564dc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:08.112046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36328ca-4407-4b42-a266-8b20caf564dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:08.131402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f36328ca-4407-4b42-a266-8b20caf564dc None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:24:08.132607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f36328ca-4407-4b42-a266-8b20caf564dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:24:08.204233 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 572/1148] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:24:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:24:08.227973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60b3f6-f875-4666-a1b2-5fc26f3b74ea None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:08.249933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:08.339864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60b3f6-f875-4666-a1b2-5fc26f3b74ea None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:08.375988 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b2310-39d7-43ea-885b-ee189e7fb9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:08.376709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2310-39d7-43ea-885b-ee189e7fb9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:08.376994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2310-39d7-43ea-885b-ee189e7fb9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:08.377272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b2310-39d7-43ea-885b-ee189e7fb9e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:08.388177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b2310-39d7-43ea-885b-ee189e7fb9e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-7a6b2310-39d7-43ea-885b-ee189e7fb9e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:24:08.395883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:24:08.396942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:08.397930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:24:08.402089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 577/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:24:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:24:08.403346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36328ca-4407-4b42-a266-8b20caf564dc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:08.509898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36328ca-4407-4b42-a266-8b20caf564dc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:08.512244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:08.546951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048a5b8-e3b6-498c-a76f-cd4bc31f8a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:08.547612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048a5b8-e3b6-498c-a76f-cd4bc31f8a86 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:08.547899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048a5b8-e3b6-498c-a76f-cd4bc31f8a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:08.548166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048a5b8-e3b6-498c-a76f-cd4bc31f8a86 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:08.548921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048a5b8-e3b6-498c-a76f-cd4bc31f8a86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:24:08.549226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048a5b8-e3b6-498c-a76f-cd4bc31f8a86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:08.550679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048a5b8-e3b6-498c-a76f-cd4bc31f8a86 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:08.559028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 573/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:24:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:24:08.567605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b2310-39d7-43ea-885b-ee189e7fb9e9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:08.664841 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:08.690145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b2310-39d7-43ea-885b-ee189e7fb9e9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:08.726021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f9b00-04cb-4a41-90f8-65c7fb139ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:08.726910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f9b00-04cb-4a41-90f8-65c7fb139ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:08.727190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f9b00-04cb-4a41-90f8-65c7fb139ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:08.727466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f9b00-04cb-4a41-90f8-65c7fb139ff8 None admin] PATH_INFO: `/v3/users/4de00d64c8e94a7982cc34a9f6b9b9d9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:08.728779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f9b00-04cb-4a41-90f8-65c7fb139ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=4de00d64c8e94a7982cc34a9f6b9b9d9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:08.730216 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f9b00-04cb-4a41-90f8-65c7fb139ff8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:08.731866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 578/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:24:08 2022] GET /identity/v3/users/4de00d64c8e94a7982cc34a9f6b9b9d9 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:08.740978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048a5b8-e3b6-498c-a76f-cd4bc31f8a86 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:08.855183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048a5b8-e3b6-498c-a76f-cd4bc31f8a86 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:08.889874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe7721-89fa-4bc3-b8c5-2fdc4e1eb02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:08.890746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe7721-89fa-4bc3-b8c5-2fdc4e1eb02b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:08.891025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe7721-89fa-4bc3-b8c5-2fdc4e1eb02b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:08.891297 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe7721-89fa-4bc3-b8c5-2fdc4e1eb02b None admin] PATH_INFO: `/v3/projects/dc8789a78400438bb213ce4017660495` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:08.892606 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe7721-89fa-4bc3-b8c5-2fdc4e1eb02b None admin] RBAC: Authorizing `identity:get_project(project_id=dc8789a78400438bb213ce4017660495)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:08.894031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe7721-89fa-4bc3-b8c5-2fdc4e1eb02b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:08.895757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 574/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:24:08 2022] GET /identity/v3/projects/dc8789a78400438bb213ce4017660495 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:08.904846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f9b00-04cb-4a41-90f8-65c7fb139ff8 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:09.007377 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f9b00-04cb-4a41-90f8-65c7fb139ff8 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:09.041800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48af01d0-70ff-4df5-b511-13074e0d0aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:09.042678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af01d0-70ff-4df5-b511-13074e0d0aa8 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:09.042988 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af01d0-70ff-4df5-b511-13074e0d0aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:09.043261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af01d0-70ff-4df5-b511-13074e0d0aa8 None admin] PATH_INFO: `/v3/projects/dc8789a78400438bb213ce4017660495/users/4de00d64c8e94a7982cc34a9f6b9b9d9/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:09.045454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af01d0-70ff-4df5-b511-13074e0d0aa8 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc8789a78400438bb213ce4017660495, user_id=4de00d64c8e94a7982cc34a9f6b9b9d9, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:09.046921 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48af01d0-70ff-4df5-b511-13074e0d0aa8 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:09.062774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 579/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:24:08 2022] PUT /identity/v3/projects/dc8789a78400438bb213ce4017660495/users/4de00d64c8e94a7982cc34a9f6b9b9d9/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:24:11.021291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d17fb9-411b-4552-87d5-a443868384ec None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:11.022188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d17fb9-411b-4552-87d5-a443868384ec None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:11.022373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d17fb9-411b-4552-87d5-a443868384ec None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:11.024565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 575/1154] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:24:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:24:11.030940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98698970-c36e-426b-b218-4d75f91fc63e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:11.031390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98698970-c36e-426b-b218-4d75f91fc63e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:11.031515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98698970-c36e-426b-b218-4d75f91fc63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:11.039345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:11.051750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-98698970-c36e-426b-b218-4d75f91fc63e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:24:11.052983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-98698970-c36e-426b-b218-4d75f91fc63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:24:11.124484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 580/1155] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:24:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:24:11.148331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d17fb9-411b-4552-87d5-a443868384ec None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:11.188648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:11.262235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d17fb9-411b-4552-87d5-a443868384ec None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:11.297537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36f2f6-6155-4aae-848c-4b04cbc2ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:11.298251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36f2f6-6155-4aae-848c-4b04cbc2ec77 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:11.298531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36f2f6-6155-4aae-848c-4b04cbc2ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:11.298837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36f2f6-6155-4aae-848c-4b04cbc2ec77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:11.309847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36f2f6-6155-4aae-848c-4b04cbc2ec77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-bd36f2f6-6155-4aae-848c-4b04cbc2ec77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:24:11.317398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:24:11.318435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:24:11.319404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 576/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:24:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:24:11.326621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98698970-c36e-426b-b218-4d75f91fc63e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:11.437773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98698970-c36e-426b-b218-4d75f91fc63e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:11.475179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90fbdd-bec0-43de-8f58-436b2b4be127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:11.475848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90fbdd-bec0-43de-8f58-436b2b4be127 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:11.476125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90fbdd-bec0-43de-8f58-436b2b4be127 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:11.476395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90fbdd-bec0-43de-8f58-436b2b4be127 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:11.477141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90fbdd-bec0-43de-8f58-436b2b4be127 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:24:11.477449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90fbdd-bec0-43de-8f58-436b2b4be127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:11.478878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90fbdd-bec0-43de-8f58-436b2b4be127 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:11.480246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:11.487831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 581/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:24:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:11.493773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36f2f6-6155-4aae-848c-4b04cbc2ec77 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:11.595011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:11.603286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36f2f6-6155-4aae-848c-4b04cbc2ec77 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:11.640502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8b4d5-38f4-498b-ae0a-888af36082ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:11.641762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b4d5-38f4-498b-ae0a-888af36082ba None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:11.642128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b4d5-38f4-498b-ae0a-888af36082ba None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:11.642473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b4d5-38f4-498b-ae0a-888af36082ba None admin] PATH_INFO: `/v3/users/4de00d64c8e94a7982cc34a9f6b9b9d9` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:11.644840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8b4d5-38f4-498b-ae0a-888af36082ba None admin] RBAC: Authorizing `identity:get_user(user_id=4de00d64c8e94a7982cc34a9f6b9b9d9)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:11.647604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8b4d5-38f4-498b-ae0a-888af36082ba None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:11.650272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 577/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:24:11 2022] GET /identity/v3/users/4de00d64c8e94a7982cc34a9f6b9b9d9 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:11.656996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90fbdd-bec0-43de-8f58-436b2b4be127 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:11.775064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90fbdd-bec0-43de-8f58-436b2b4be127 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:11.810907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d606068-9d31-4b52-892a-c0159436f491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:11.811545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d606068-9d31-4b52-892a-c0159436f491 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:11.811835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d606068-9d31-4b52-892a-c0159436f491 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:11.812104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d606068-9d31-4b52-892a-c0159436f491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:11.877639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:11.886139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d606068-9d31-4b52-892a-c0159436f491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:11.887635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d606068-9d31-4b52-892a-c0159436f491 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:11.989582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 582/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:24:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:24:11.996317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8b4d5-38f4-498b-ae0a-888af36082ba None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:12.115857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8b4d5-38f4-498b-ae0a-888af36082ba None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:12.150677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a23e8-57a2-4acc-ad39-a5160a66e079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:12.151496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a23e8-57a2-4acc-ad39-a5160a66e079 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:12.151773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a23e8-57a2-4acc-ad39-a5160a66e079 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:12.152040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a23e8-57a2-4acc-ad39-a5160a66e079 None admin] PATH_INFO: `/v3/projects/dc8789a78400438bb213ce4017660495` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:12.153969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a23e8-57a2-4acc-ad39-a5160a66e079 None admin] RBAC: Authorizing `identity:get_project(project_id=dc8789a78400438bb213ce4017660495)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:12.156357 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a23e8-57a2-4acc-ad39-a5160a66e079 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:12.159156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 578/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:24:11 2022] GET /identity/v3/projects/dc8789a78400438bb213ce4017660495 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:12.168632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d606068-9d31-4b52-892a-c0159436f491 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:12.271160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:12.281372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d606068-9d31-4b52-892a-c0159436f491 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:12.326211 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bced4-c528-43c5-a18c-5a0d73419f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:12.326890 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bced4-c528-43c5-a18c-5a0d73419f13 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:12.327174 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bced4-c528-43c5-a18c-5a0d73419f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:12.327416 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bced4-c528-43c5-a18c-5a0d73419f13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:12.328272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bced4-c528-43c5-a18c-5a0d73419f13 None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=dc8789a78400438bb213ce4017660495, user.id=4de00d64c8e94a7982cc34a9f6b9b9d9) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:24:12.328585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bced4-c528-43c5-a18c-5a0d73419f13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:12.330029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bced4-c528-43c5-a18c-5a0d73419f13 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:12.336561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 583/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:24:12 2022] GET /identity/v3/role_assignments?user.id=4de00d64c8e94a7982cc34a9f6b9b9d9&scope.project.id=dc8789a78400438bb213ce4017660495&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:14.355252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c5ee9-7925-4827-bedc-b1f2810045dc None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:14.355851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c5ee9-7925-4827-bedc-b1f2810045dc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:14.355851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c5ee9-7925-4827-bedc-b1f2810045dc None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:14.357199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 579/1162] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:24:14.363673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb31893-868c-416b-8132-3d9c1e6c1d1d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:14.364389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb31893-868c-416b-8132-3d9c1e6c1d1d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:14.364543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb31893-868c-416b-8132-3d9c1e6c1d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:14.393242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9cb31893-868c-416b-8132-3d9c1e6c1d1d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:24:14.395098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9cb31893-868c-416b-8132-3d9c1e6c1d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:24:14.465266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 584/1163] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:24:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:24:14.487821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c5ee9-7925-4827-bedc-b1f2810045dc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:14.523105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:14.600872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c5ee9-7925-4827-bedc-b1f2810045dc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:14.635676 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f186910-2ea7-421a-9fa8-dc06e998a106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:14.636245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f186910-2ea7-421a-9fa8-dc06e998a106 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:14.636527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f186910-2ea7-421a-9fa8-dc06e998a106 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:14.636795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f186910-2ea7-421a-9fa8-dc06e998a106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:14.707768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f186910-2ea7-421a-9fa8-dc06e998a106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:14.709212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f186910-2ea7-421a-9fa8-dc06e998a106 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:14.738904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:14.809427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 580/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:24:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:24:14.816288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb31893-868c-416b-8132-3d9c1e6c1d1d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:14.931358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:14.941759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb31893-868c-416b-8132-3d9c1e6c1d1d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:14.976824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76714b-68dd-49d8-a0d6-2f34d484ad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:14.977619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76714b-68dd-49d8-a0d6-2f34d484ad67 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:14.977895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76714b-68dd-49d8-a0d6-2f34d484ad67 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:14.978166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76714b-68dd-49d8-a0d6-2f34d484ad67 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:14.984913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76714b-68dd-49d8-a0d6-2f34d484ad67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:14.986354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76714b-68dd-49d8-a0d6-2f34d484ad67 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-0a76714b-68dd-49d8-a0d6-2f34d484ad67 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:24:15.000554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:24:15.001720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:24:15.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 585/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jun 29 08:24:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:24:15.011097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f186910-2ea7-421a-9fa8-dc06e998a106 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:15.126948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f186910-2ea7-421a-9fa8-dc06e998a106 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:15.156404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:15.164706 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dba9c6-119f-48f9-bfe8-18bcb360933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:15.165360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dba9c6-119f-48f9-bfe8-18bcb360933d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:15.165633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dba9c6-119f-48f9-bfe8-18bcb360933d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:15.165906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dba9c6-119f-48f9-bfe8-18bcb360933d None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:15.166636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dba9c6-119f-48f9-bfe8-18bcb360933d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:24:15.166972 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dba9c6-119f-48f9-bfe8-18bcb360933d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:15.168343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dba9c6-119f-48f9-bfe8-18bcb360933d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:15.176291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 581/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Wed Jun 29 08:24:15 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:17.186677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d9119-91a4-4616-9aab-25349880f6a2 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:17.187291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d9119-91a4-4616-9aab-25349880f6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:17.187291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d9119-91a4-4616-9aab-25349880f6a2 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:17.188647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 586/1167] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:24:17.195148 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90978db0-1f53-4542-aa2b-473ef0ccd36d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:17.195862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90978db0-1f53-4542-aa2b-473ef0ccd36d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:17.196031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90978db0-1f53-4542-aa2b-473ef0ccd36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:17.219405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-90978db0-1f53-4542-aa2b-473ef0ccd36d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:24:17.220597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-90978db0-1f53-4542-aa2b-473ef0ccd36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:24:17.304971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 582/1168] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:24:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:24:17.328755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d9119-91a4-4616-9aab-25349880f6a2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:17.435767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d9119-91a4-4616-9aab-25349880f6a2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:17.472336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad5867-36c0-42b9-970d-7c335b4d033b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:17.473002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad5867-36c0-42b9-970d-7c335b4d033b None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:17.473282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad5867-36c0-42b9-970d-7c335b4d033b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:17.473553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad5867-36c0-42b9-970d-7c335b4d033b None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:17.474303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad5867-36c0-42b9-970d-7c335b4d033b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:17.475712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad5867-36c0-42b9-970d-7c335b4d033b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:17.495407 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 587/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Wed Jun 29 08:24:17 2022] POST /identity/v3/domains => generated 267 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:24:19.518332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038992f-b156-4814-b1c0-d800b7e4d655 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:19.519537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038992f-b156-4814-b1c0-d800b7e4d655 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:19.519537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038992f-b156-4814-b1c0-d800b7e4d655 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:19.521585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 583/1170] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:24:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:24:19.527902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e2b71-d326-4343-93fa-8cb3a51e5274 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:19.528351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e2b71-d326-4343-93fa-8cb3a51e5274 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:19.528460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5e2b71-d326-4343-93fa-8cb3a51e5274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:19.554471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9e5e2b71-d326-4343-93fa-8cb3a51e5274 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:24:19.555693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9e5e2b71-d326-4343-93fa-8cb3a51e5274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:24:19.558532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:19.636617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 588/1171] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:24:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:24:19.661321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038992f-b156-4814-b1c0-d800b7e4d655 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:19.773108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038992f-b156-4814-b1c0-d800b7e4d655 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:19.797009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:19.809309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a36e9-6ca4-4371-85f2-03e928da21b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:19.810166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a36e9-6ca4-4371-85f2-03e928da21b5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:19.810447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a36e9-6ca4-4371-85f2-03e928da21b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:19.810716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a36e9-6ca4-4371-85f2-03e928da21b5 None admin] PATH_INFO: `/v3/domains/87c940cdd16e466cad5f4c447c889d2c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:19.821490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a36e9-6ca4-4371-85f2-03e928da21b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=87c940cdd16e466cad5f4c447c889d2c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:19.822996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a36e9-6ca4-4371-85f2-03e928da21b5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:19.824647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 584/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Wed Jun 29 08:24:19 2022] GET /identity/v3/domains/87c940cdd16e466cad5f4c447c889d2c => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:19.834751 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e2b71-d326-4343-93fa-8cb3a51e5274 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:19.947123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5e2b71-d326-4343-93fa-8cb3a51e5274 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:19.984142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79521ded-f520-4fd7-974f-bcb26ef74254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:19.984807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79521ded-f520-4fd7-974f-bcb26ef74254 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:19.985090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79521ded-f520-4fd7-974f-bcb26ef74254 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:19.985366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79521ded-f520-4fd7-974f-bcb26ef74254 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:19.986073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79521ded-f520-4fd7-974f-bcb26ef74254 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:19.987718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79521ded-f520-4fd7-974f-bcb26ef74254 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:19.997603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:20.020614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 589/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:24:19 2022] POST /identity/v3/projects => generated 361 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:24:22.068341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39765e-78e9-4c20-a5fc-632937254a5a None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:22.069210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39765e-78e9-4c20-a5fc-632937254a5a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:22.069210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39765e-78e9-4c20-a5fc-632937254a5a None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:22.071503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 585/1174] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:24:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:24:22.080164 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbb3bb-7e1e-4c9f-b6d5-67cdb831c86c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:22.080608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbb3bb-7e1e-4c9f-b6d5-67cdb831c86c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:22.080713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbb3bb-7e1e-4c9f-b6d5-67cdb831c86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:22.104751 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a2dbb3bb-7e1e-4c9f-b6d5-67cdb831c86c None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:24:22.105940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a2dbb3bb-7e1e-4c9f-b6d5-67cdb831c86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:24:22.176452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 590/1175] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:24:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:24:22.209452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39765e-78e9-4c20-a5fc-632937254a5a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:22.319441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39765e-78e9-4c20-a5fc-632937254a5a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:22.355052 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336be95-417e-422a-94a3-2d8df6cd6265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:22.356377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336be95-417e-422a-94a3-2d8df6cd6265 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:22.356807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336be95-417e-422a-94a3-2d8df6cd6265 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:22.357104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336be95-417e-422a-94a3-2d8df6cd6265 None admin] PATH_INFO: `/v3/domains/87c940cdd16e466cad5f4c447c889d2c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:22.359189 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336be95-417e-422a-94a3-2d8df6cd6265 None admin] RBAC: Authorizing `identity:get_domain(domain_id=87c940cdd16e466cad5f4c447c889d2c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:22.361453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336be95-417e-422a-94a3-2d8df6cd6265 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:22.364476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 586/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Wed Jun 29 08:24:22 2022] GET /identity/v3/domains/87c940cdd16e466cad5f4c447c889d2c => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:22.373590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbb3bb-7e1e-4c9f-b6d5-67cdb831c86c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:22.481925 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbb3bb-7e1e-4c9f-b6d5-67cdb831c86c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:22.506451 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:22.519990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742081fc-aac9-4806-b419-6b37a299c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:22.520657 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742081fc-aac9-4806-b419-6b37a299c077 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:22.520936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742081fc-aac9-4806-b419-6b37a299c077 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:22.521205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742081fc-aac9-4806-b419-6b37a299c077 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:22.521917 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742081fc-aac9-4806-b419-6b37a299c077 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:22.523400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742081fc-aac9-4806-b419-6b37a299c077 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:22.545009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 591/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:24:22 2022] POST /identity/v3/users => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:24:24.587621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da61ca-e55c-40ee-a839-b0b555ef0eed None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:24.588899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da61ca-e55c-40ee-a839-b0b555ef0eed None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:24.588899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da61ca-e55c-40ee-a839-b0b555ef0eed None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:24.591184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 587/1178] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:24:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:24:24.597785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541f18-6bfa-417a-9cb7-1edd3da5bc32 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:24.598225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541f18-6bfa-417a-9cb7-1edd3da5bc32 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:24.598324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4541f18-6bfa-417a-9cb7-1edd3da5bc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:24.626572 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b4541f18-6bfa-417a-9cb7-1edd3da5bc32 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:24:24.628587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b4541f18-6bfa-417a-9cb7-1edd3da5bc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:24:24.709795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 592/1179] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:24:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:24:24.733388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da61ca-e55c-40ee-a839-b0b555ef0eed None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:24.794426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:24.843427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da61ca-e55c-40ee-a839-b0b555ef0eed None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:24.881658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd5604-fad7-4bdc-b030-50ed9e658a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:24.882435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd5604-fad7-4bdc-b030-50ed9e658a88 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:24.882718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd5604-fad7-4bdc-b030-50ed9e658a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:24.883012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd5604-fad7-4bdc-b030-50ed9e658a88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:24.894885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd5604-fad7-4bdc-b030-50ed9e658a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-78bd5604-fad7-4bdc-b030-50ed9e658a88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:24:24.902345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:24:24.903502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:24:24.904569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 588/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:24:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:24:24.909750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541f18-6bfa-417a-9cb7-1edd3da5bc32 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:25.009920 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4541f18-6bfa-417a-9cb7-1edd3da5bc32 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:25.018923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:25.046534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173193d-4dcf-4d1c-8c1f-356e69b5785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:25.047233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173193d-4dcf-4d1c-8c1f-356e69b5785e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:25.047495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173193d-4dcf-4d1c-8c1f-356e69b5785e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:25.047768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173193d-4dcf-4d1c-8c1f-356e69b5785e None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:25.048504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173193d-4dcf-4d1c-8c1f-356e69b5785e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:24:25.048806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173193d-4dcf-4d1c-8c1f-356e69b5785e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:25.050191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173193d-4dcf-4d1c-8c1f-356e69b5785e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:25.058174 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 593/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:24:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:25.064252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd5604-fad7-4bdc-b030-50ed9e658a88 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:25.170471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd5604-fad7-4bdc-b030-50ed9e658a88 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:25.183957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:25.205472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d01428-c5af-4183-8f3b-a1d7e9e3cfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:25.206978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d01428-c5af-4183-8f3b-a1d7e9e3cfcc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:25.207272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d01428-c5af-4183-8f3b-a1d7e9e3cfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:25.207556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d01428-c5af-4183-8f3b-a1d7e9e3cfcc None admin] PATH_INFO: `/v3/users/84966abadb494bce952659750f16fe50` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:25.221362 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d01428-c5af-4183-8f3b-a1d7e9e3cfcc None admin] RBAC: Authorizing `identity:get_user(user_id=84966abadb494bce952659750f16fe50)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:25.222884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d01428-c5af-4183-8f3b-a1d7e9e3cfcc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:25.224537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 589/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:24:25 2022] GET /identity/v3/users/84966abadb494bce952659750f16fe50 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:25.230622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173193d-4dcf-4d1c-8c1f-356e69b5785e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:25.336981 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173193d-4dcf-4d1c-8c1f-356e69b5785e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:25.372960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5261f-c3cd-45c3-99b1-0d992c0cdd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:25.373595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5261f-c3cd-45c3-99b1-0d992c0cdd3b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:25.373879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5261f-c3cd-45c3-99b1-0d992c0cdd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:25.374154 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5261f-c3cd-45c3-99b1-0d992c0cdd3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:25.418105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:25.448443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5261f-c3cd-45c3-99b1-0d992c0cdd3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:25.449913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5261f-c3cd-45c3-99b1-0d992c0cdd3b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:25.557753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 594/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:24:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:24:25.564132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d01428-c5af-4183-8f3b-a1d7e9e3cfcc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:25.671088 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d01428-c5af-4183-8f3b-a1d7e9e3cfcc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:25.711906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34837cb-14c9-4cab-a024-c1c3fbe88f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:25.712786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34837cb-14c9-4cab-a024-c1c3fbe88f0f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:25.713069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34837cb-14c9-4cab-a024-c1c3fbe88f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:25.713377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34837cb-14c9-4cab-a024-c1c3fbe88f0f None admin] PATH_INFO: `/v3/projects/6b9fd6d3f70f4e1e8355dfbf2fa869e7` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:25.714956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34837cb-14c9-4cab-a024-c1c3fbe88f0f None admin] RBAC: Authorizing `identity:get_project(project_id=6b9fd6d3f70f4e1e8355dfbf2fa869e7)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:25.716408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34837cb-14c9-4cab-a024-c1c3fbe88f0f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:25.718146 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 590/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:24:25 2022] GET /identity/v3/projects/6b9fd6d3f70f4e1e8355dfbf2fa869e7 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:25.724448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5261f-c3cd-45c3-99b1-0d992c0cdd3b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:25.797398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:25.830306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5261f-c3cd-45c3-99b1-0d992c0cdd3b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:25.868736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f468a0-3aa1-4362-b170-94143e042215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:25.869380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f468a0-3aa1-4362-b170-94143e042215 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:25.869662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f468a0-3aa1-4362-b170-94143e042215 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:25.869942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f468a0-3aa1-4362-b170-94143e042215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:25.870801 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f468a0-3aa1-4362-b170-94143e042215 None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=6b9fd6d3f70f4e1e8355dfbf2fa869e7, user.id=84966abadb494bce952659750f16fe50) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:24:25.871123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f468a0-3aa1-4362-b170-94143e042215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:25.872566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f468a0-3aa1-4362-b170-94143e042215 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:25.878934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 595/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:24:25 2022] GET /identity/v3/role_assignments?user.id=84966abadb494bce952659750f16fe50&scope.project.id=6b9fd6d3f70f4e1e8355dfbf2fa869e7&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:27.850318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da9321-803f-4474-af2c-b9cabb91b9b1 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:27.851247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da9321-803f-4474-af2c-b9cabb91b9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:27.851247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da9321-803f-4474-af2c-b9cabb91b9b1 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:27.853515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 591/1186] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:24:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:24:27.862316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138fb9c-ca74-4898-86c1-5e117f1b3d2f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:27.863258 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138fb9c-ca74-4898-86c1-5e117f1b3d2f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:27.863547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138fb9c-ca74-4898-86c1-5e117f1b3d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:27.883409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c138fb9c-ca74-4898-86c1-5e117f1b3d2f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:24:27.884610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c138fb9c-ca74-4898-86c1-5e117f1b3d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:24:27.957994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 596/1187] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:24:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:24:27.981884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da9321-803f-4474-af2c-b9cabb91b9b1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:28.031552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:28.106918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da9321-803f-4474-af2c-b9cabb91b9b1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:28.142639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a1e70-cd4e-42a4-9ed4-ea18ff966c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:28.143454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a1e70-cd4e-42a4-9ed4-ea18ff966c55 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:28.143829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a1e70-cd4e-42a4-9ed4-ea18ff966c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:28.144190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a1e70-cd4e-42a4-9ed4-ea18ff966c55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:28.154661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a1e70-cd4e-42a4-9ed4-ea18ff966c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5d9a1e70-cd4e-42a4-9ed4-ea18ff966c55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:24:28.162245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:24:28.163329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:24:28.164327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 592/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:24:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:24:28.172630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138fb9c-ca74-4898-86c1-5e117f1b3d2f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:28.254358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:28.297344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138fb9c-ca74-4898-86c1-5e117f1b3d2f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:28.333053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5595eb-1473-4859-a619-f5bfd3b1bb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:28.333717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5595eb-1473-4859-a619-f5bfd3b1bb25 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:28.333998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5595eb-1473-4859-a619-f5bfd3b1bb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:28.334271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5595eb-1473-4859-a619-f5bfd3b1bb25 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:28.335064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5595eb-1473-4859-a619-f5bfd3b1bb25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:24:28.335370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5595eb-1473-4859-a619-f5bfd3b1bb25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:28.336746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5595eb-1473-4859-a619-f5bfd3b1bb25 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:28.345208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 597/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:24:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:28.354055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a1e70-cd4e-42a4-9ed4-ea18ff966c55 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:28.463131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:28.467785 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a1e70-cd4e-42a4-9ed4-ea18ff966c55 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:28.502535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23cb31-6ec2-4e69-b023-af1e43f3cfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:28.503434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cb31-6ec2-4e69-b023-af1e43f3cfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:28.503709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cb31-6ec2-4e69-b023-af1e43f3cfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:28.503980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cb31-6ec2-4e69-b023-af1e43f3cfe2 None admin] PATH_INFO: `/v3/users/84966abadb494bce952659750f16fe50` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:28.505201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23cb31-6ec2-4e69-b023-af1e43f3cfe2 None admin] RBAC: Authorizing `identity:get_user(user_id=84966abadb494bce952659750f16fe50)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:28.506636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23cb31-6ec2-4e69-b023-af1e43f3cfe2 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:28.508298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 593/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:24:28 2022] GET /identity/v3/users/84966abadb494bce952659750f16fe50 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:28.514532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5595eb-1473-4859-a619-f5bfd3b1bb25 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:28.620087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5595eb-1473-4859-a619-f5bfd3b1bb25 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:28.645746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:28.654735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f1435-3d9d-4018-8078-b28793d11053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:28.655588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1435-3d9d-4018-8078-b28793d11053 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:28.655868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1435-3d9d-4018-8078-b28793d11053 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:28.656138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f1435-3d9d-4018-8078-b28793d11053 None admin] PATH_INFO: `/v3/projects/6b9fd6d3f70f4e1e8355dfbf2fa869e7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:28.657320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f1435-3d9d-4018-8078-b28793d11053 None admin] RBAC: Authorizing `identity:get_project(project_id=6b9fd6d3f70f4e1e8355dfbf2fa869e7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:28.658798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f1435-3d9d-4018-8078-b28793d11053 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:28.660535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 598/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:24:28 2022] GET /identity/v3/projects/6b9fd6d3f70f4e1e8355dfbf2fa869e7 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:24:28.666830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23cb31-6ec2-4e69-b023-af1e43f3cfe2 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:28.769028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23cb31-6ec2-4e69-b023-af1e43f3cfe2 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:28.804196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3a530-27cb-4bf9-866d-2b8b240499c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:28.805071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a530-27cb-4bf9-866d-2b8b240499c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:28.805346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a530-27cb-4bf9-866d-2b8b240499c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:28.805615 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a530-27cb-4bf9-866d-2b8b240499c3 None admin] PATH_INFO: `/v3/projects/6b9fd6d3f70f4e1e8355dfbf2fa869e7/users/84966abadb494bce952659750f16fe50/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:28.807826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3a530-27cb-4bf9-866d-2b8b240499c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b9fd6d3f70f4e1e8355dfbf2fa869e7, user_id=84966abadb494bce952659750f16fe50, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:28.809245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3a530-27cb-4bf9-866d-2b8b240499c3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:28.824917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 594/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:24:28 2022] PUT /identity/v3/projects/6b9fd6d3f70f4e1e8355dfbf2fa869e7/users/84966abadb494bce952659750f16fe50/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:24:30.806442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a20fb3-c5d3-4190-9070-f5b44b151857 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:30.807037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a20fb3-c5d3-4190-9070-f5b44b151857 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:30.807037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a20fb3-c5d3-4190-9070-f5b44b151857 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:30.808310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 599/1193] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:24:30.814615 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2624c-6ae3-488c-9e43-e6aba5543e27 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:30.815114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2624c-6ae3-488c-9e43-e6aba5543e27 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:30.815208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2624c-6ae3-488c-9e43-e6aba5543e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:30.840154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c3c2624c-6ae3-488c-9e43-e6aba5543e27 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:24:30.841371 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c3c2624c-6ae3-488c-9e43-e6aba5543e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:24:30.864840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:30.916838 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 595/1194] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:24:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:24:30.944247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a20fb3-c5d3-4190-9070-f5b44b151857 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:31.054061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a20fb3-c5d3-4190-9070-f5b44b151857 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:31.089821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a8f8-9cea-4024-b86e-f9ec8a995f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:31.090609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a8f8-9cea-4024-b86e-f9ec8a995f57 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:31.090921 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a8f8-9cea-4024-b86e-f9ec8a995f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:31.091195 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4a8f8-9cea-4024-b86e-f9ec8a995f57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:31.102149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4a8f8-9cea-4024-b86e-f9ec8a995f57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-37a4a8f8-9cea-4024-b86e-f9ec8a995f57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:24:31.109708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:24:31.110792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:24:31.111775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 600/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:24:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:24:31.118132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2624c-6ae3-488c-9e43-e6aba5543e27 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:31.237880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2624c-6ae3-488c-9e43-e6aba5543e27 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:31.277398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ee958-6b5d-45b3-8a83-c2c60c4c9c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:31.278057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ee958-6b5d-45b3-8a83-c2c60c4c9c13 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:31.278343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ee958-6b5d-45b3-8a83-c2c60c4c9c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:31.278620 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ee958-6b5d-45b3-8a83-c2c60c4c9c13 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:31.279402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ee958-6b5d-45b3-8a83-c2c60c4c9c13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:24:31.279714 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ee958-6b5d-45b3-8a83-c2c60c4c9c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:31.281086 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ee958-6b5d-45b3-8a83-c2c60c4c9c13 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:31.289878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 596/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:24:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:31.296109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a8f8-9cea-4024-b86e-f9ec8a995f57 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:31.355023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:31.412916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4a8f8-9cea-4024-b86e-f9ec8a995f57 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:31.448402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7d574-a764-4597-ac8d-934544ce5811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:31.449268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7d574-a764-4597-ac8d-934544ce5811 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:31.449549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7d574-a764-4597-ac8d-934544ce5811 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:31.449829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7d574-a764-4597-ac8d-934544ce5811 None admin] PATH_INFO: `/v3/users/84966abadb494bce952659750f16fe50` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:31.451072 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7d574-a764-4597-ac8d-934544ce5811 None admin] RBAC: Authorizing `identity:get_user(user_id=84966abadb494bce952659750f16fe50)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:31.452563 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7d574-a764-4597-ac8d-934544ce5811 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:31.454148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 601/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Wed Jun 29 08:24:31 2022] GET /identity/v3/users/84966abadb494bce952659750f16fe50 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:31.460452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ee958-6b5d-45b3-8a83-c2c60c4c9c13 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:31.481179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:31.568708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ee958-6b5d-45b3-8a83-c2c60c4c9c13 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:31.602866 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661b11-6a43-4e93-a951-e564bca1d82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:31.603500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661b11-6a43-4e93-a951-e564bca1d82a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:31.603782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661b11-6a43-4e93-a951-e564bca1d82a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:31.604055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661b11-6a43-4e93-a951-e564bca1d82a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:31.673483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87661b11-6a43-4e93-a951-e564bca1d82a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:31.675004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87661b11-6a43-4e93-a951-e564bca1d82a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:31.693399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:31.774403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 597/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:24:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:24:31.781345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7d574-a764-4597-ac8d-934544ce5811 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:31.889281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7d574-a764-4597-ac8d-934544ce5811 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:31.912251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:31.924831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb0fe4-19a6-4484-907d-656543ee2044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:31.925775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb0fe4-19a6-4484-907d-656543ee2044 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:31.926076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb0fe4-19a6-4484-907d-656543ee2044 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:31.926333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb0fe4-19a6-4484-907d-656543ee2044 None admin] PATH_INFO: `/v3/projects/6b9fd6d3f70f4e1e8355dfbf2fa869e7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:31.927581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb0fe4-19a6-4484-907d-656543ee2044 None admin] RBAC: Authorizing `identity:get_project(project_id=6b9fd6d3f70f4e1e8355dfbf2fa869e7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:31.929013 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb0fe4-19a6-4484-907d-656543ee2044 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:31.930707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 602/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Wed Jun 29 08:24:31 2022] GET /identity/v3/projects/6b9fd6d3f70f4e1e8355dfbf2fa869e7 => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:31.937822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661b11-6a43-4e93-a951-e564bca1d82a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:24:32.049698 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661b11-6a43-4e93-a951-e564bca1d82a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:24:32.067419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:32.086270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982dff-4843-4028-bb16-2ff7a642ee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:24:32.086953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982dff-4843-4028-bb16-2ff7a642ee1a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:24:32.087235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982dff-4843-4028-bb16-2ff7a642ee1a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:24:32.087509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982dff-4843-4028-bb16-2ff7a642ee1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:24:32.088342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38982dff-4843-4028-bb16-2ff7a642ee1a None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=6b9fd6d3f70f4e1e8355dfbf2fa869e7, user.id=84966abadb494bce952659750f16fe50) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:24:32.089001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38982dff-4843-4028-bb16-2ff7a642ee1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:24:32.090450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38982dff-4843-4028-bb16-2ff7a642ee1a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:24:32.096438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 598/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:24:31 2022] GET /identity/v3/role_assignments?user.id=84966abadb494bce952659750f16fe50&scope.project.id=6b9fd6d3f70f4e1e8355dfbf2fa869e7&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:24:41.920633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:24:42.094058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=9 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:28.053107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb395c5-ff9b-489b-9b90-0f128c4ef163 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:28.053107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb395c5-ff9b-489b-9b90-0f128c4ef163 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:28.053107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb395c5-ff9b-489b-9b90-0f128c4ef163 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:28.054383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 603/1201] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:25:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:25:28.058542 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e363e-83c3-48af-9b12-32e007d5d3a1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:28.058974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e363e-83c3-48af-9b12-32e007d5d3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:28.060510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e363e-83c3-48af-9b12-32e007d5d3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:28.077298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5e6e363e-83c3-48af-9b12-32e007d5d3a1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:25:28.078519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5e6e363e-83c3-48af-9b12-32e007d5d3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:25:28.155632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 599/1202] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:25:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:25:28.180955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb395c5-ff9b-489b-9b90-0f128c4ef163 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:28.305095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb395c5-ff9b-489b-9b90-0f128c4ef163 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:28.344885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab90dfb-3c99-46c8-ba8c-172361e3a4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:28.345667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab90dfb-3c99-46c8-ba8c-172361e3a4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:28.345977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab90dfb-3c99-46c8-ba8c-172361e3a4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:28.346246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab90dfb-3c99-46c8-ba8c-172361e3a4b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:28.358536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab90dfb-3c99-46c8-ba8c-172361e3a4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:28.360127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab90dfb-3c99-46c8-ba8c-172361e3a4b3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-3ab90dfb-3c99-46c8-ba8c-172361e3a4b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:28.379671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:28.381199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:28.382156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:28.382156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:28.382156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:28.382156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:28.382156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 604/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:28.387425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e363e-83c3-48af-9b12-32e007d5d3a1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:28.510627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e363e-83c3-48af-9b12-32e007d5d3a1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:28.542152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:28.549610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba09392-99a3-43e7-ba76-4ad8c285a051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:28.550447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba09392-99a3-43e7-ba76-4ad8c285a051 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:28.550828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba09392-99a3-43e7-ba76-4ad8c285a051 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:28.551190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba09392-99a3-43e7-ba76-4ad8c285a051 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:28.552061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba09392-99a3-43e7-ba76-4ad8c285a051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:28.552386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba09392-99a3-43e7-ba76-4ad8c285a051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:28.553964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba09392-99a3-43e7-ba76-4ad8c285a051 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:28.567456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 600/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:28.573632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab90dfb-3c99-46c8-ba8c-172361e3a4b3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:28.644730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:28.711648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab90dfb-3c99-46c8-ba8c-172361e3a4b3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:28.754645 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6feadd-1f9f-490c-a2ca-113384206a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:28.755914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6feadd-1f9f-490c-a2ca-113384206a0f None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:28.756248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6feadd-1f9f-490c-a2ca-113384206a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:28.756529 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6feadd-1f9f-490c-a2ca-113384206a0f None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:28.757322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6feadd-1f9f-490c-a2ca-113384206a0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:28.758788 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6feadd-1f9f-490c-a2ca-113384206a0f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:28.782728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 605/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Wed Jun 29 08:25:28 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:25:30.935843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a493ebc-fc61-411a-8341-7cb9d3798788 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:30.936467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a493ebc-fc61-411a-8341-7cb9d3798788 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:30.936467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a493ebc-fc61-411a-8341-7cb9d3798788 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:30.937890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 601/1206] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:25:30.944886 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ddda7-aa95-4e87-addc-29fb6aae3026 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:30.945531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ddda7-aa95-4e87-addc-29fb6aae3026 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:30.945531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ddda7-aa95-4e87-addc-29fb6aae3026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:30.965896 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-078ddda7-aa95-4e87-addc-29fb6aae3026 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:25:30.967167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-078ddda7-aa95-4e87-addc-29fb6aae3026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:25:31.044625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 606/1207] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:25:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:25:31.069581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a493ebc-fc61-411a-8341-7cb9d3798788 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:31.196974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a493ebc-fc61-411a-8341-7cb9d3798788 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:31.233035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950e683-b943-461a-9da1-bee087f7bc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:31.233908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950e683-b943-461a-9da1-bee087f7bc8f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:31.234196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950e683-b943-461a-9da1-bee087f7bc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:31.234473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950e683-b943-461a-9da1-bee087f7bc8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:31.246266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950e683-b943-461a-9da1-bee087f7bc8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-6950e683-b943-461a-9da1-bee087f7bc8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:25:31.253956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:31.255221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:25:31.256178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:31.265795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 602/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:25:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:25:31.265971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ddda7-aa95-4e87-addc-29fb6aae3026 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:31.304696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:31.373218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ddda7-aa95-4e87-addc-29fb6aae3026 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:31.408982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbd21f-2d24-423d-9aba-28eede7bb55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:31.409649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbd21f-2d24-423d-9aba-28eede7bb55f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:31.409936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbd21f-2d24-423d-9aba-28eede7bb55f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:31.410210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbd21f-2d24-423d-9aba-28eede7bb55f None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:31.410994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfbd21f-2d24-423d-9aba-28eede7bb55f None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:31.411388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfbd21f-2d24-423d-9aba-28eede7bb55f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:31.412702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfbd21f-2d24-423d-9aba-28eede7bb55f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:31.421258 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 607/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:25:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:31.427770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950e683-b943-461a-9da1-bee087f7bc8f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:31.488990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:31.544001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950e683-b943-461a-9da1-bee087f7bc8f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:31.580589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4e574-1f32-4687-b022-60378a49a215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:31.581456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4e574-1f32-4687-b022-60378a49a215 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:31.581739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4e574-1f32-4687-b022-60378a49a215 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:31.582010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4e574-1f32-4687-b022-60378a49a215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:31.594994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4e574-1f32-4687-b022-60378a49a215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:31.596520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4e574-1f32-4687-b022-60378a49a215 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-4ec4e574-1f32-4687-b022-60378a49a215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:31.614670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:31.617034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:31.618154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:31.618154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:31.618154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:31.618154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:31.618154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 603/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:31.621728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbd21f-2d24-423d-9aba-28eede7bb55f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:31.732591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:31.735591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbd21f-2d24-423d-9aba-28eede7bb55f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:31.773381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3245f-b0d2-4343-a409-fb71534ae51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:31.774099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3245f-b0d2-4343-a409-fb71534ae51a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:31.774403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3245f-b0d2-4343-a409-fb71534ae51a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:31.774678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3245f-b0d2-4343-a409-fb71534ae51a None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:31.775449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3245f-b0d2-4343-a409-fb71534ae51a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:31.775759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3245f-b0d2-4343-a409-fb71534ae51a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:31.777119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3245f-b0d2-4343-a409-fb71534ae51a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:31.790296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 608/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:31.796037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4e574-1f32-4687-b022-60378a49a215 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:31.908895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4e574-1f32-4687-b022-60378a49a215 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:31.910538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:31.947868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e3024-5624-4706-bd02-92a93059aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:31.949995 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e3024-5624-4706-bd02-92a93059aadb None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:31.949995 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e3024-5624-4706-bd02-92a93059aadb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:31.949995 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e3024-5624-4706-bd02-92a93059aadb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:31.957584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e3024-5624-4706-bd02-92a93059aadb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:31.959307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5e3024-5624-4706-bd02-92a93059aadb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5f5e3024-5624-4706-bd02-92a93059aadb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:25:31.976642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:31.977854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:31.980248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:31.980248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:31.980248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:25:31.980248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:25:31.980248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:25:31.980248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:25:31.980248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:25:31.980248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:25:31.980248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:31.980248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:31.980248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 604/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:25:31.984029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3245f-b0d2-4343-a409-fb71534ae51a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:32.109656 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3245f-b0d2-4343-a409-fb71534ae51a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:32.151705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc326d3-a95a-4db6-b0ca-3bd5f4f496b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:32.152410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc326d3-a95a-4db6-b0ca-3bd5f4f496b3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:32.152695 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc326d3-a95a-4db6-b0ca-3bd5f4f496b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:32.152969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc326d3-a95a-4db6-b0ca-3bd5f4f496b3 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:32.153834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc326d3-a95a-4db6-b0ca-3bd5f4f496b3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:32.154147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc326d3-a95a-4db6-b0ca-3bd5f4f496b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:32.155645 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc326d3-a95a-4db6-b0ca-3bd5f4f496b3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:32.177879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 609/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jun 29 08:25:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:32.183769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e3024-5624-4706-bd02-92a93059aadb None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:32.305530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e3024-5624-4706-bd02-92a93059aadb None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:32.343043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d56f5f-e909-4537-aa8f-b69d7d927afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:32.343918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d56f5f-e909-4537-aa8f-b69d7d927afc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:32.345191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d56f5f-e909-4537-aa8f-b69d7d927afc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:32.345191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d56f5f-e909-4537-aa8f-b69d7d927afc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:32.385856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:32.428726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d56f5f-e909-4537-aa8f-b69d7d927afc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:32.430236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d56f5f-e909-4537-aa8f-b69d7d927afc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:32.550400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 605/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:25:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:25:32.556774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc326d3-a95a-4db6-b0ca-3bd5f4f496b3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:32.583415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:32.668916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc326d3-a95a-4db6-b0ca-3bd5f4f496b3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:32.705744 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a379e41-c9ad-42e6-9452-8b36744b5b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:32.706547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a379e41-c9ad-42e6-9452-8b36744b5b77 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:32.706868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a379e41-c9ad-42e6-9452-8b36744b5b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:32.707143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a379e41-c9ad-42e6-9452-8b36744b5b77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:32.718374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a379e41-c9ad-42e6-9452-8b36744b5b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:32.719906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a379e41-c9ad-42e6-9452-8b36744b5b77 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9a379e41-c9ad-42e6-9452-8b36744b5b77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:32.737962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:32.739134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:32.740204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:32.740204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:32.740204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:32.740204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:32.740204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 610/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:32.745702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d56f5f-e909-4537-aa8f-b69d7d927afc None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:32.811162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:32.853425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d56f5f-e909-4537-aa8f-b69d7d927afc None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:32.889116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd0525-32dc-4066-9c91-d63cc7991264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:32.889932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd0525-32dc-4066-9c91-d63cc7991264 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:32.890212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd0525-32dc-4066-9c91-d63cc7991264 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:32.890483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd0525-32dc-4066-9c91-d63cc7991264 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:32.891244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd0525-32dc-4066-9c91-d63cc7991264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:32.891548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd0525-32dc-4066-9c91-d63cc7991264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:32.892931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd0525-32dc-4066-9c91-d63cc7991264 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:32.905006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 606/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:32.910863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a379e41-c9ad-42e6-9452-8b36744b5b77 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:32.991132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:33.033671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a379e41-c9ad-42e6-9452-8b36744b5b77 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:33.073233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5e997-7ebb-4811-b0bd-bc14e7a4eb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:33.074158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5e997-7ebb-4811-b0bd-bc14e7a4eb3e None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:33.074552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5e997-7ebb-4811-b0bd-bc14e7a4eb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:33.074948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5e997-7ebb-4811-b0bd-bc14e7a4eb3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:33.085496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5e997-7ebb-4811-b0bd-bc14e7a4eb3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:33.087437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5e997-7ebb-4811-b0bd-bc14e7a4eb3e None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-4df5e997-7ebb-4811-b0bd-bc14e7a4eb3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:25:33.102312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:33.103506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:33.104654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:25:33.104654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:25:33.104654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:25:33.104654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:25:33.104654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:25:33.104654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:25:33.104654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:33.104654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:33.105153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 611/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:25:33.110778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd0525-32dc-4066-9c91-d63cc7991264 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:33.242962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd0525-32dc-4066-9c91-d63cc7991264 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:33.270349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:33.284450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ecd14-0f59-415f-b8b3-36b4dc918871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:33.285156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ecd14-0f59-415f-b8b3-36b4dc918871 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:33.285451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ecd14-0f59-415f-b8b3-36b4dc918871 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:33.285745 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ecd14-0f59-415f-b8b3-36b4dc918871 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:33.286679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ecd14-0f59-415f-b8b3-36b4dc918871 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:33.287019 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ecd14-0f59-415f-b8b3-36b4dc918871 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:33.288510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ecd14-0f59-415f-b8b3-36b4dc918871 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:33.302606 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 607/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:25:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:33.309334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5e997-7ebb-4811-b0bd-bc14e7a4eb3e None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:33.437038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5e997-7ebb-4811-b0bd-bc14e7a4eb3e None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:33.479206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:33.485334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce174f-8f1a-4f5d-9ff5-48adeacaedc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:33.486001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce174f-8f1a-4f5d-9ff5-48adeacaedc3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:33.486282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce174f-8f1a-4f5d-9ff5-48adeacaedc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:33.486743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce174f-8f1a-4f5d-9ff5-48adeacaedc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:33.487646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce174f-8f1a-4f5d-9ff5-48adeacaedc3 None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=00a08f31638d4eaab5ca83d6f493db6c) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:33.487943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce174f-8f1a-4f5d-9ff5-48adeacaedc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:33.489397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce174f-8f1a-4f5d-9ff5-48adeacaedc3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:33.506723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 612/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:25:33 2022] GET /identity/v3/role_assignments?user.id=00a08f31638d4eaab5ca83d6f493db6c&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:35.669761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6a11f-ba5c-47b6-958d-675f759a2bdd None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:35.670318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6a11f-ba5c-47b6-958d-675f759a2bdd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:35.670318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6a11f-ba5c-47b6-958d-675f759a2bdd None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:35.671802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 608/1220] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:25:35.677223 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d757251-b7b5-4ae3-b475-619f196e9e1e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:35.677726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d757251-b7b5-4ae3-b475-619f196e9e1e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:35.677909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d757251-b7b5-4ae3-b475-619f196e9e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:35.698030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1d757251-b7b5-4ae3-b475-619f196e9e1e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:25:35.699313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1d757251-b7b5-4ae3-b475-619f196e9e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:25:35.774378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 613/1221] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:25:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:25:35.797311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6a11f-ba5c-47b6-958d-675f759a2bdd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:35.904119 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6a11f-ba5c-47b6-958d-675f759a2bdd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:35.940225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f017aa-2a5a-4d37-b384-96d2227d2d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:35.941121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f017aa-2a5a-4d37-b384-96d2227d2d6c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:35.941398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f017aa-2a5a-4d37-b384-96d2227d2d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:35.941670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f017aa-2a5a-4d37-b384-96d2227d2d6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:35.954111 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f017aa-2a5a-4d37-b384-96d2227d2d6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-28f017aa-2a5a-4d37-b384-96d2227d2d6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:25:35.961903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:35.963092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:35.964288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 609/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:25:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:35.971964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d757251-b7b5-4ae3-b475-619f196e9e1e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:36.081705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d757251-b7b5-4ae3-b475-619f196e9e1e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:36.106667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:36.117649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a5e62-e926-40fe-bf2a-87b3092f9988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:36.118322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a5e62-e926-40fe-bf2a-87b3092f9988 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:36.118602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a5e62-e926-40fe-bf2a-87b3092f9988 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:36.118927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a5e62-e926-40fe-bf2a-87b3092f9988 None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:36.119672 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a5e62-e926-40fe-bf2a-87b3092f9988 None admin] RBAC: Adding query filter params (name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:36.119976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a5e62-e926-40fe-bf2a-87b3092f9988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:36.121344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a5e62-e926-40fe-bf2a-87b3092f9988 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:36.129431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 614/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:25:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:36.135624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f017aa-2a5a-4d37-b384-96d2227d2d6c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:36.179713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:36.246370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f017aa-2a5a-4d37-b384-96d2227d2d6c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:36.286616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a49fe-1aa9-46eb-b6ce-a82b4455cb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:36.287599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a49fe-1aa9-46eb-b6ce-a82b4455cb64 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:36.287864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a49fe-1aa9-46eb-b6ce-a82b4455cb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:36.288137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a49fe-1aa9-46eb-b6ce-a82b4455cb64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:36.302235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a49fe-1aa9-46eb-b6ce-a82b4455cb64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:36.303975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a49fe-1aa9-46eb-b6ce-a82b4455cb64 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-a43a49fe-1aa9-46eb-b6ce-a82b4455cb64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:36.324747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:36.325875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:36.326849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:36.326849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:36.326849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:36.326849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:36.326849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 610/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:36.332087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a5e62-e926-40fe-bf2a-87b3092f9988 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:36.442474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a5e62-e926-40fe-bf2a-87b3092f9988 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:36.482012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c115d0-20ea-44a4-8fef-c55321f97388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:36.482834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c115d0-20ea-44a4-8fef-c55321f97388 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:36.483220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c115d0-20ea-44a4-8fef-c55321f97388 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:36.483617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c115d0-20ea-44a4-8fef-c55321f97388 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:36.484495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c115d0-20ea-44a4-8fef-c55321f97388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:36.484817 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c115d0-20ea-44a4-8fef-c55321f97388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:36.486385 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c115d0-20ea-44a4-8fef-c55321f97388 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:36.502560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 615/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:36.509434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a49fe-1aa9-46eb-b6ce-a82b4455cb64 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:36.615669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:36.623306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a49fe-1aa9-46eb-b6ce-a82b4455cb64 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:36.661712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085de84-fc72-45ff-8af6-967ac1118b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:36.662584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085de84-fc72-45ff-8af6-967ac1118b5e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:36.662968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085de84-fc72-45ff-8af6-967ac1118b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:36.663241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085de84-fc72-45ff-8af6-967ac1118b5e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:36.671883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085de84-fc72-45ff-8af6-967ac1118b5e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:36.673667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1085de84-fc72-45ff-8af6-967ac1118b5e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-1085de84-fc72-45ff-8af6-967ac1118b5e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:25:36.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:36.690106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:36.691097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:36.691097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:36.691097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:25:36.691097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:25:36.691097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:25:36.691097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:25:36.691097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:25:36.691097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:25:36.691097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:36.691097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:36.691097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 611/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:25:36.696490 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c115d0-20ea-44a4-8fef-c55321f97388 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:36.748229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:36.807575 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c115d0-20ea-44a4-8fef-c55321f97388 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:36.850947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b063c4a-eb82-475b-8150-72bbc2c2dfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:36.851619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b063c4a-eb82-475b-8150-72bbc2c2dfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:36.851902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b063c4a-eb82-475b-8150-72bbc2c2dfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:36.852168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b063c4a-eb82-475b-8150-72bbc2c2dfb5 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:36.853061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b063c4a-eb82-475b-8150-72bbc2c2dfb5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:36.853368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b063c4a-eb82-475b-8150-72bbc2c2dfb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:36.854864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b063c4a-eb82-475b-8150-72bbc2c2dfb5 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:36.872597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 616/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jun 29 08:25:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:36.878199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085de84-fc72-45ff-8af6-967ac1118b5e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:36.982195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085de84-fc72-45ff-8af6-967ac1118b5e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:37.020198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c9aa4-7c6e-4776-8413-6dd38863e105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:37.021182 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c9aa4-7c6e-4776-8413-6dd38863e105 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:37.021649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c9aa4-7c6e-4776-8413-6dd38863e105 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:37.021939 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c9aa4-7c6e-4776-8413-6dd38863e105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:37.032977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c9aa4-7c6e-4776-8413-6dd38863e105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:37.034444 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c9aa4-7c6e-4776-8413-6dd38863e105 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-857c9aa4-7c6e-4776-8413-6dd38863e105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:37.053422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:37.054500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:37.055633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:37.055633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:37.055633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:37.055633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:37.055633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 612/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:37.060520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b063c4a-eb82-475b-8150-72bbc2c2dfb5 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:37.169756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b063c4a-eb82-475b-8150-72bbc2c2dfb5 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:37.172146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:37.207692 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc80c2f-66d8-421d-83c6-23a2647e98ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:37.208917 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc80c2f-66d8-421d-83c6-23a2647e98ad None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:37.209202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc80c2f-66d8-421d-83c6-23a2647e98ad None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:37.209493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc80c2f-66d8-421d-83c6-23a2647e98ad None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:37.210297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc80c2f-66d8-421d-83c6-23a2647e98ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:37.210660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc80c2f-66d8-421d-83c6-23a2647e98ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:37.212160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc80c2f-66d8-421d-83c6-23a2647e98ad None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:37.227050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 617/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:37.233433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c9aa4-7c6e-4776-8413-6dd38863e105 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:37.243351 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:37.340070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c9aa4-7c6e-4776-8413-6dd38863e105 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:37.374773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4df32-277d-4b40-96c3-ce80bd440d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:37.375577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4df32-277d-4b40-96c3-ce80bd440d4e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:37.375886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4df32-277d-4b40-96c3-ce80bd440d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:37.376155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4df32-277d-4b40-96c3-ce80bd440d4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:37.383671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4df32-277d-4b40-96c3-ce80bd440d4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:37.385329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4df32-277d-4b40-96c3-ce80bd440d4e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-1ff4df32-277d-4b40-96c3-ce80bd440d4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:25:37.401829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:37.402910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:37.403993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:25:37.403993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:25:37.403993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:25:37.403993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:25:37.403993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:25:37.403993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:25:37.403993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:37.403993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:37.408314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 613/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:37.409766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc80c2f-66d8-421d-83c6-23a2647e98ad None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:37.517737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc80c2f-66d8-421d-83c6-23a2647e98ad None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:37.555982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e8469-73f2-4f30-ae4f-fe50ca443e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:37.556677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e8469-73f2-4f30-ae4f-fe50ca443e4d None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:37.556957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e8469-73f2-4f30-ae4f-fe50ca443e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:37.557235 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e8469-73f2-4f30-ae4f-fe50ca443e4d None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:37.558037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e8469-73f2-4f30-ae4f-fe50ca443e4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:37.558346 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e8469-73f2-4f30-ae4f-fe50ca443e4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:37.559848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e8469-73f2-4f30-ae4f-fe50ca443e4d None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:37.577274 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 618/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:25:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:37.584605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4df32-277d-4b40-96c3-ce80bd440d4e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:37.659509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:37.713356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4df32-277d-4b40-96c3-ce80bd440d4e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:37.750574 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e1f8f-3482-4cf7-be6e-66849c943ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:37.751511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e1f8f-3482-4cf7-be6e-66849c943ff5 None admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:37.751792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e1f8f-3482-4cf7-be6e-66849c943ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:37.752067 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e1f8f-3482-4cf7-be6e-66849c943ff5 None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/00a08f31638d4eaab5ca83d6f493db6c/roles/8f88a54639c64acdbdb152911e5f9eb2` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:37.766940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e1f8f-3482-4cf7-be6e-66849c943ff5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=00a08f31638d4eaab5ca83d6f493db6c, role_id=8f88a54639c64acdbdb152911e5f9eb2)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:37.768631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e1f8f-3482-4cf7-be6e-66849c943ff5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:37.785506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 614/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:25:37 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/00a08f31638d4eaab5ca83d6f493db6c/roles/8f88a54639c64acdbdb152911e5f9eb2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:25:39.860830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee876c-c5d3-4820-80c2-e1c4d10bdbd3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:39.861426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee876c-c5d3-4820-80c2-e1c4d10bdbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:39.861523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee876c-c5d3-4820-80c2-e1c4d10bdbd3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:39.863108 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 619/1233] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:25:39.868717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2d96f-69b2-452e-b165-064c7effd194 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:39.869144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2d96f-69b2-452e-b165-064c7effd194 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:39.869269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2d96f-69b2-452e-b165-064c7effd194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:39.886128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-99e2d96f-69b2-452e-b165-064c7effd194 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:25:39.887346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-99e2d96f-69b2-452e-b165-064c7effd194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:25:39.962394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 615/1234] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:25:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:25:39.986083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee876c-c5d3-4820-80c2-e1c4d10bdbd3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:40.012720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:40.100979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee876c-c5d3-4820-80c2-e1c4d10bdbd3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:40.137596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707dac3-c9b8-432c-b5c3-94f3aa49c7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:40.138245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707dac3-c9b8-432c-b5c3-94f3aa49c7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:40.138527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707dac3-c9b8-432c-b5c3-94f3aa49c7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:40.138861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707dac3-c9b8-432c-b5c3-94f3aa49c7d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:40.149609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707dac3-c9b8-432c-b5c3-94f3aa49c7d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-2707dac3-c9b8-432c-b5c3-94f3aa49c7d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:25:40.157463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:40.158819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:40.159964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 620/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Wed Jun 29 08:25:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:40.165054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2d96f-69b2-452e-b165-064c7effd194 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:40.219938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:40.279182 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2d96f-69b2-452e-b165-064c7effd194 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:40.315116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed746ca-445a-4280-af5c-bb70e7fbfa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:40.315796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed746ca-445a-4280-af5c-bb70e7fbfa6a None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:40.316078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed746ca-445a-4280-af5c-bb70e7fbfa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:40.316351 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed746ca-445a-4280-af5c-bb70e7fbfa6a None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:40.317107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed746ca-445a-4280-af5c-bb70e7fbfa6a None admin] RBAC: Adding query filter params (name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:40.317495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed746ca-445a-4280-af5c-bb70e7fbfa6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:40.319354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed746ca-445a-4280-af5c-bb70e7fbfa6a None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:40.327764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 616/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Wed Jun 29 08:25:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:40.334899 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707dac3-c9b8-432c-b5c3-94f3aa49c7d9 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:40.423292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:40.438935 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707dac3-c9b8-432c-b5c3-94f3aa49c7d9 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:40.476995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ad3b2-8605-48a3-bc3e-a9676f6c2e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:40.477872 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ad3b2-8605-48a3-bc3e-a9676f6c2e48 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:40.478148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ad3b2-8605-48a3-bc3e-a9676f6c2e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:40.478418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ad3b2-8605-48a3-bc3e-a9676f6c2e48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:40.489103 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ad3b2-8605-48a3-bc3e-a9676f6c2e48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:40.490553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ad3b2-8605-48a3-bc3e-a9676f6c2e48 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-314ad3b2-8605-48a3-bc3e-a9676f6c2e48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:40.507667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:40.508852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:40.510043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:40.510043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:40.510043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:40.510043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:40.510043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 621/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:25:40.514840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed746ca-445a-4280-af5c-bb70e7fbfa6a None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:40.618963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed746ca-445a-4280-af5c-bb70e7fbfa6a None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:40.628024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:40.658156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c7b54-9c66-4f36-9c71-4f7b97033c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:40.659212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c7b54-9c66-4f36-9c71-4f7b97033c41 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:40.659462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c7b54-9c66-4f36-9c71-4f7b97033c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:40.659837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c7b54-9c66-4f36-9c71-4f7b97033c41 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:40.660557 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c7b54-9c66-4f36-9c71-4f7b97033c41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:40.660955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c7b54-9c66-4f36-9c71-4f7b97033c41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:40.662813 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3c7b54-9c66-4f36-9c71-4f7b97033c41 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:40.679011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 617/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:40.681212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ad3b2-8605-48a3-bc3e-a9676f6c2e48 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:40.791908 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ad3b2-8605-48a3-bc3e-a9676f6c2e48 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:40.828815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d8831-cf77-4d35-9fbb-9a6e8a4b7d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:40.829626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d8831-cf77-4d35-9fbb-9a6e8a4b7d56 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:40.829912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d8831-cf77-4d35-9fbb-9a6e8a4b7d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:40.830195 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d8831-cf77-4d35-9fbb-9a6e8a4b7d56 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:40.833232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:40.839067 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d8831-cf77-4d35-9fbb-9a6e8a4b7d56 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:40.840759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d8831-cf77-4d35-9fbb-9a6e8a4b7d56 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-604d8831-cf77-4d35-9fbb-9a6e8a4b7d56 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:25:40.855977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:40.857149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:40.858078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:40.858078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:40.858078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:25:40.858078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:25:40.858078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:25:40.858078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:25:40.858078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:25:40.858078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:25:40.858078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:40.858078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:40.858078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 622/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:40.863219 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c7b54-9c66-4f36-9c71-4f7b97033c41 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:40.968224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c7b54-9c66-4f36-9c71-4f7b97033c41 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:41.006645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bd7b6-3b26-40c1-b49c-bb1cbe90c886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:41.007394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bd7b6-3b26-40c1-b49c-bb1cbe90c886 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:41.007681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bd7b6-3b26-40c1-b49c-bb1cbe90c886 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:41.008001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bd7b6-3b26-40c1-b49c-bb1cbe90c886 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:41.008855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bd7b6-3b26-40c1-b49c-bb1cbe90c886 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:41.009167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bd7b6-3b26-40c1-b49c-bb1cbe90c886 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:41.010642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bd7b6-3b26-40c1-b49c-bb1cbe90c886 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:41.028823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 618/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jun 29 08:25:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:25:41.034511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d8831-cf77-4d35-9fbb-9a6e8a4b7d56 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:41.145317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d8831-cf77-4d35-9fbb-9a6e8a4b7d56 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:41.184115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa7237-d2a2-426f-bb12-19cb17aae2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:41.184781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa7237-d2a2-426f-bb12-19cb17aae2dd None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:41.185069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa7237-d2a2-426f-bb12-19cb17aae2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:41.185348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa7237-d2a2-426f-bb12-19cb17aae2dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:41.256520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:41.264211 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa7237-d2a2-426f-bb12-19cb17aae2dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:41.265767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa7237-d2a2-426f-bb12-19cb17aae2dd None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:41.372969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 623/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:25:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:25:41.378977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bd7b6-3b26-40c1-b49c-bb1cbe90c886 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:41.412464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:41.491885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bd7b6-3b26-40c1-b49c-bb1cbe90c886 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:41.528579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9065d16-fa08-4591-8466-2d21cbd99f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:41.529378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9065d16-fa08-4591-8466-2d21cbd99f9f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:41.529654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9065d16-fa08-4591-8466-2d21cbd99f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:41.529996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9065d16-fa08-4591-8466-2d21cbd99f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:41.541197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9065d16-fa08-4591-8466-2d21cbd99f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:41.542844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9065d16-fa08-4591-8466-2d21cbd99f9f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-e9065d16-fa08-4591-8466-2d21cbd99f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:41.561155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:41.562346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:41.563310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:41.563310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:41.563310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:41.563310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:41.563310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 619/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:41.568831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa7237-d2a2-426f-bb12-19cb17aae2dd None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:41.668465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:41.676099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa7237-d2a2-426f-bb12-19cb17aae2dd None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:41.714581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fff68-0b0f-4348-997c-54f16d6cdf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:41.715263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fff68-0b0f-4348-997c-54f16d6cdf03 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:41.715546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fff68-0b0f-4348-997c-54f16d6cdf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:41.715816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fff68-0b0f-4348-997c-54f16d6cdf03 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:41.716567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52fff68-0b0f-4348-997c-54f16d6cdf03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:41.716874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52fff68-0b0f-4348-997c-54f16d6cdf03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:41.718422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52fff68-0b0f-4348-997c-54f16d6cdf03 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:41.731726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 624/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:25:41.737346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9065d16-fa08-4591-8466-2d21cbd99f9f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:41.824443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:41.848554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9065d16-fa08-4591-8466-2d21cbd99f9f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:41.885884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac886b5-b5e0-4e13-914f-b19598fba3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:41.886659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac886b5-b5e0-4e13-914f-b19598fba3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:41.887091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac886b5-b5e0-4e13-914f-b19598fba3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:41.887365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac886b5-b5e0-4e13-914f-b19598fba3c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:41.894609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac886b5-b5e0-4e13-914f-b19598fba3c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:41.896161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac886b5-b5e0-4e13-914f-b19598fba3c5 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-4ac886b5-b5e0-4e13-914f-b19598fba3c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:25:41.912438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:41.913527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:41.919500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:25:41.919500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:25:41.919500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:25:41.919500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:25:41.919500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:25:41.919500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:25:41.919500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:41.919500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:41.919500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 620/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:25:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:41.920095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fff68-0b0f-4348-997c-54f16d6cdf03 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:42.031216 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fff68-0b0f-4348-997c-54f16d6cdf03 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:42.067325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21498b-f661-442a-9590-c32dc61d8253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:42.068008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21498b-f661-442a-9590-c32dc61d8253 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:42.068250 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21498b-f661-442a-9590-c32dc61d8253 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:42.068519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21498b-f661-442a-9590-c32dc61d8253 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:42.069305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21498b-f661-442a-9590-c32dc61d8253 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:42.069632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21498b-f661-442a-9590-c32dc61d8253 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:42.071097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21498b-f661-442a-9590-c32dc61d8253 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:42.084631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 625/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:25:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:42.091382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac886b5-b5e0-4e13-914f-b19598fba3c5 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:42.205100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac886b5-b5e0-4e13-914f-b19598fba3c5 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:42.246612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43487677-5b52-48c5-b593-711105c7efe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:42.247476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487677-5b52-48c5-b593-711105c7efe0 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:42.247764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487677-5b52-48c5-b593-711105c7efe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:42.248035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43487677-5b52-48c5-b593-711105c7efe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:42.249050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43487677-5b52-48c5-b593-711105c7efe0 None admin] RBAC: Adding query filter params (role.id=8f88a54639c64acdbdb152911e5f9eb2, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=00a08f31638d4eaab5ca83d6f493db6c) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:42.249369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43487677-5b52-48c5-b593-711105c7efe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:42.250916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43487677-5b52-48c5-b593-711105c7efe0 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:42.257942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 621/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:25:42 2022] GET /identity/v3/role_assignments?user.id=00a08f31638d4eaab5ca83d6f493db6c&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=8f88a54639c64acdbdb152911e5f9eb2 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:44.396611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d6a10-d795-45a1-93c5-fb1533da8df5 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:44.397324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d6a10-d795-45a1-93c5-fb1533da8df5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:44.397324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d6a10-d795-45a1-93c5-fb1533da8df5 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:44.398965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 626/1247] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:25:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:25:44.406407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03855-0cde-494b-a3f1-6c1c87c080d6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:44.406900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03855-0cde-494b-a3f1-6c1c87c080d6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:44.406990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03855-0cde-494b-a3f1-6c1c87c080d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:44.412381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:44.426483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-73a03855-0cde-494b-a3f1-6c1c87c080d6 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:25:44.427780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-73a03855-0cde-494b-a3f1-6c1c87c080d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:25:44.505903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 622/1248] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:25:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:25:44.532579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d6a10-d795-45a1-93c5-fb1533da8df5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:44.551583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:44.646527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d6a10-d795-45a1-93c5-fb1533da8df5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:44.687027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333c1ca-6230-4d97-ad23-3585c1f09740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:44.687754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333c1ca-6230-4d97-ad23-3585c1f09740 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:44.688042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333c1ca-6230-4d97-ad23-3585c1f09740 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:44.688316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333c1ca-6230-4d97-ad23-3585c1f09740 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:44.698581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333c1ca-6230-4d97-ad23-3585c1f09740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9333c1ca-6230-4d97-ad23-3585c1f09740 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:25:44.706135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:44.707340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:44.708945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 627/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:25:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:44.714545 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a03855-0cde-494b-a3f1-6c1c87c080d6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:44.820404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a03855-0cde-494b-a3f1-6c1c87c080d6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:44.852553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:44.860773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17c3d2-36c2-46b6-9adc-a7d2e5434fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:44.861441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17c3d2-36c2-46b6-9adc-a7d2e5434fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:44.862014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17c3d2-36c2-46b6-9adc-a7d2e5434fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:44.862645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17c3d2-36c2-46b6-9adc-a7d2e5434fa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:44.864130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17c3d2-36c2-46b6-9adc-a7d2e5434fa1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:44.864655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17c3d2-36c2-46b6-9adc-a7d2e5434fa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:44.866524 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17c3d2-36c2-46b6-9adc-a7d2e5434fa1 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:44.876133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 623/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:25:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:25:44.882528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333c1ca-6230-4d97-ad23-3585c1f09740 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:44.968821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:44.994274 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333c1ca-6230-4d97-ad23-3585c1f09740 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:45.031728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb76e5-b67d-41f8-a8ad-b3bb98421e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:45.032523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb76e5-b67d-41f8-a8ad-b3bb98421e23 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:45.032800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb76e5-b67d-41f8-a8ad-b3bb98421e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:45.033064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb76e5-b67d-41f8-a8ad-b3bb98421e23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:45.043845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb76e5-b67d-41f8-a8ad-b3bb98421e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:45.045342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb76e5-b67d-41f8-a8ad-b3bb98421e23 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ebdb76e5-b67d-41f8-a8ad-b3bb98421e23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:45.064102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:45.065517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:45.066633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:45.066633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:45.066633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:45.066633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:45.072347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 628/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:45.075504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17c3d2-36c2-46b6-9adc-a7d2e5434fa1 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:45.192097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17c3d2-36c2-46b6-9adc-a7d2e5434fa1 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:45.235772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca3a9b-934f-4f62-9b92-58670a6cb387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:45.236507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3a9b-934f-4f62-9b92-58670a6cb387 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:45.236953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3a9b-934f-4f62-9b92-58670a6cb387 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:45.237263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca3a9b-934f-4f62-9b92-58670a6cb387 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:45.238124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca3a9b-934f-4f62-9b92-58670a6cb387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:45.238534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca3a9b-934f-4f62-9b92-58670a6cb387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:45.240371 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca3a9b-934f-4f62-9b92-58670a6cb387 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:45.257461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 624/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:45.263542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb76e5-b67d-41f8-a8ad-b3bb98421e23 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:45.379025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb76e5-b67d-41f8-a8ad-b3bb98421e23 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:45.412905 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:45.422748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd92eb-fcc0-497c-b576-caef2c7dbaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:45.423681 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd92eb-fcc0-497c-b576-caef2c7dbaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:45.423991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd92eb-fcc0-497c-b576-caef2c7dbaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:45.424393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd92eb-fcc0-497c-b576-caef2c7dbaf1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:45.433384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd92eb-fcc0-497c-b576-caef2c7dbaf1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:45.435009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd92eb-fcc0-497c-b576-caef2c7dbaf1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-9fbd92eb-fcc0-497c-b576-caef2c7dbaf1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:25:45.452165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:45.453146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:45.454130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:45.454130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:45.454130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:25:45.454130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:25:45.454130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:25:45.454130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:25:45.454130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:25:45.454130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:25:45.454130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:45.454130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:45.454130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 629/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:45.460861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca3a9b-934f-4f62-9b92-58670a6cb387 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:45.525267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:45.575210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca3a9b-934f-4f62-9b92-58670a6cb387 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:45.616422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2120b-4398-48bc-913b-bbd3d2952a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:45.617161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2120b-4398-48bc-913b-bbd3d2952a05 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:45.617473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2120b-4398-48bc-913b-bbd3d2952a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:45.617846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2120b-4398-48bc-913b-bbd3d2952a05 None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:45.618735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2120b-4398-48bc-913b-bbd3d2952a05 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:45.619073 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2120b-4398-48bc-913b-bbd3d2952a05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:45.620660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2120b-4398-48bc-913b-bbd3d2952a05 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:45.640159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 625/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jun 29 08:25:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:45.646480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd92eb-fcc0-497c-b576-caef2c7dbaf1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:45.761480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd92eb-fcc0-497c-b576-caef2c7dbaf1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:45.805600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e3ac-f197-4db9-9292-9e10e41b219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:45.806338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e3ac-f197-4db9-9292-9e10e41b219a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:45.806625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e3ac-f197-4db9-9292-9e10e41b219a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:45.806955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13e3ac-f197-4db9-9292-9e10e41b219a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:45.871409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:45.889341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13e3ac-f197-4db9-9292-9e10e41b219a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:45.891921 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13e3ac-f197-4db9-9292-9e10e41b219a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:46.022904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 630/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:25:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 08:25:46.029726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2120b-4398-48bc-913b-bbd3d2952a05 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:46.152279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2120b-4398-48bc-913b-bbd3d2952a05 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:46.167733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:46.193543 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cf38e-73f4-4fe0-a221-7f096bf07c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:46.194619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf38e-73f4-4fe0-a221-7f096bf07c6d None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:46.194912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf38e-73f4-4fe0-a221-7f096bf07c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:46.195180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf38e-73f4-4fe0-a221-7f096bf07c6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:46.208335 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cf38e-73f4-4fe0-a221-7f096bf07c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:46.210030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cf38e-73f4-4fe0-a221-7f096bf07c6d None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-773cf38e-73f4-4fe0-a221-7f096bf07c6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:46.231584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:46.232716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:46.233759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:46.233759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:46.233759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:46.233759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:46.238001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 626/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:46.239030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e3ac-f197-4db9-9292-9e10e41b219a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:46.355401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:46.368792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13e3ac-f197-4db9-9292-9e10e41b219a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:46.407336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adcc5d-c707-4542-9e52-4bd61add14c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:46.407939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adcc5d-c707-4542-9e52-4bd61add14c3 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:46.408207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adcc5d-c707-4542-9e52-4bd61add14c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:46.408470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adcc5d-c707-4542-9e52-4bd61add14c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:46.409186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adcc5d-c707-4542-9e52-4bd61add14c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:46.409498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adcc5d-c707-4542-9e52-4bd61add14c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:46.410900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adcc5d-c707-4542-9e52-4bd61add14c3 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:46.425907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 631/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:25:46.432316 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cf38e-73f4-4fe0-a221-7f096bf07c6d None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:46.549691 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cf38e-73f4-4fe0-a221-7f096bf07c6d None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:46.589855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70175b1-aa8c-46d1-919b-e60c57d859cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:46.590780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70175b1-aa8c-46d1-919b-e60c57d859cf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:46.591050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70175b1-aa8c-46d1-919b-e60c57d859cf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:46.591318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70175b1-aa8c-46d1-919b-e60c57d859cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:46.599079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70175b1-aa8c-46d1-919b-e60c57d859cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:46.600314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70175b1-aa8c-46d1-919b-e60c57d859cf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-f70175b1-aa8c-46d1-919b-e60c57d859cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:25:46.616760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:46.617845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:46.618832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:25:46.618832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:25:46.618832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:25:46.618832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:25:46.618832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:25:46.618832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:25:46.618832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:46.618832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:46.618832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 627/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:25:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:46.624272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adcc5d-c707-4542-9e52-4bd61add14c3 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:46.729516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adcc5d-c707-4542-9e52-4bd61add14c3 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:46.767212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47be7c-d793-4427-aeef-0d340d390e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:46.767931 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47be7c-d793-4427-aeef-0d340d390e03 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:46.768188 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47be7c-d793-4427-aeef-0d340d390e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:46.768507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47be7c-d793-4427-aeef-0d340d390e03 None admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:46.769382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47be7c-d793-4427-aeef-0d340d390e03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:46.769697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47be7c-d793-4427-aeef-0d340d390e03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:46.771345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47be7c-d793-4427-aeef-0d340d390e03 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:46.784762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 632/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:25:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:46.791809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70175b1-aa8c-46d1-919b-e60c57d859cf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:46.819435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:46.903003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70175b1-aa8c-46d1-919b-e60c57d859cf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:46.941980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e492480-cf1a-4940-b4a1-7253721a54c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:46.942965 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e492480-cf1a-4940-b4a1-7253721a54c9 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:46.943289 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e492480-cf1a-4940-b4a1-7253721a54c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:46.943598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e492480-cf1a-4940-b4a1-7253721a54c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:46.944538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e492480-cf1a-4940-b4a1-7253721a54c9 None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=00a08f31638d4eaab5ca83d6f493db6c) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:46.944866 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e492480-cf1a-4940-b4a1-7253721a54c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:46.946367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e492480-cf1a-4940-b4a1-7253721a54c9 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:46.952907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 628/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:25:46 2022] GET /identity/v3/role_assignments?user.id=00a08f31638d4eaab5ca83d6f493db6c&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:25:49.088017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1cac8-b178-4e05-a910-6cb2173694f2 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:49.088586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1cac8-b178-4e05-a910-6cb2173694f2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:49.088586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1cac8-b178-4e05-a910-6cb2173694f2 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:49.090505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 633/1261] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:25:49.096023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422361b5-98bf-4879-95fe-6db9c2993a22 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:49.096546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422361b5-98bf-4879-95fe-6db9c2993a22 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:49.096546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422361b5-98bf-4879-95fe-6db9c2993a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:49.114177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-422361b5-98bf-4879-95fe-6db9c2993a22 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:25:49.115417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-422361b5-98bf-4879-95fe-6db9c2993a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:25:49.188378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 629/1262] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:25:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:25:49.212578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1cac8-b178-4e05-a910-6cb2173694f2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:49.226086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:49.324175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1cac8-b178-4e05-a910-6cb2173694f2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:49.359810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7df55-6473-491d-8073-caf09d6c96d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:49.360524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7df55-6473-491d-8073-caf09d6c96d2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:49.360807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7df55-6473-491d-8073-caf09d6c96d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:49.361082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7df55-6473-491d-8073-caf09d6c96d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:49.371712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f7df55-6473-491d-8073-caf09d6c96d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-c5f7df55-6473-491d-8073-caf09d6c96d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:25:49.379296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:49.380747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:49.381587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 634/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:25:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:49.386795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422361b5-98bf-4879-95fe-6db9c2993a22 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:49.407963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:49.492636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422361b5-98bf-4879-95fe-6db9c2993a22 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:49.526722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177e751-7d43-4c9e-8ea7-295eeb89c88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:49.527416 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177e751-7d43-4c9e-8ea7-295eeb89c88f None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:49.527700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177e751-7d43-4c9e-8ea7-295eeb89c88f None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:49.527957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177e751-7d43-4c9e-8ea7-295eeb89c88f None admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:49.528716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177e751-7d43-4c9e-8ea7-295eeb89c88f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:49.529019 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177e751-7d43-4c9e-8ea7-295eeb89c88f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:49.530371 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177e751-7d43-4c9e-8ea7-295eeb89c88f None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:49.539166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 630/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:25:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:49.545055 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7df55-6473-491d-8073-caf09d6c96d2 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:49.618068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:49.656690 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f7df55-6473-491d-8073-caf09d6c96d2 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:49.691355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833b715-4912-4ab4-9761-0ee3c12ce1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:49.692190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833b715-4912-4ab4-9761-0ee3c12ce1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:49.692523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833b715-4912-4ab4-9761-0ee3c12ce1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:49.692901 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833b715-4912-4ab4-9761-0ee3c12ce1d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:49.703779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833b715-4912-4ab4-9761-0ee3c12ce1d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:49.705221 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833b715-4912-4ab4-9761-0ee3c12ce1d1 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-5833b715-4912-4ab4-9761-0ee3c12ce1d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:49.722721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:49.724565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:49.725686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:49.725686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:49.725686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:49.725686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:49.725686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 635/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:49.730779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177e751-7d43-4c9e-8ea7-295eeb89c88f None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:49.827187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:49.854402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177e751-7d43-4c9e-8ea7-295eeb89c88f None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:49.886633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784eb15c-5fdb-4a2b-846e-5832fcb93c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:49.887266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784eb15c-5fdb-4a2b-846e-5832fcb93c23 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:49.887545 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784eb15c-5fdb-4a2b-846e-5832fcb93c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:49.887819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784eb15c-5fdb-4a2b-846e-5832fcb93c23 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:49.888546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784eb15c-5fdb-4a2b-846e-5832fcb93c23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:49.888853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784eb15c-5fdb-4a2b-846e-5832fcb93c23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:49.890230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784eb15c-5fdb-4a2b-846e-5832fcb93c23 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:49.906897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 631/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:49.916330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833b715-4912-4ab4-9761-0ee3c12ce1d1 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:50.042187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833b715-4912-4ab4-9761-0ee3c12ce1d1 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:50.062985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:50.078914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984ae2a-7206-4fd7-b8a3-e19f507c542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:50.079714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984ae2a-7206-4fd7-b8a3-e19f507c542f None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:50.080002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984ae2a-7206-4fd7-b8a3-e19f507c542f None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:50.080276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984ae2a-7206-4fd7-b8a3-e19f507c542f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:50.088276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984ae2a-7206-4fd7-b8a3-e19f507c542f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:50.090091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984ae2a-7206-4fd7-b8a3-e19f507c542f None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-1984ae2a-7206-4fd7-b8a3-e19f507c542f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:25:50.104316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:50.105376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:50.106325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:50.106325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:50.106325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:25:50.106325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:25:50.106325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:25:50.106325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:25:50.106325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:25:50.106325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:25:50.106325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:50.106325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:50.106325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 636/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:50.111746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784eb15c-5fdb-4a2b-846e-5832fcb93c23 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:50.227620 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784eb15c-5fdb-4a2b-846e-5832fcb93c23 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:50.276511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02545f2b-ebc3-45fd-8494-3fd11e8a034c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:50.277193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02545f2b-ebc3-45fd-8494-3fd11e8a034c None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:50.277471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02545f2b-ebc3-45fd-8494-3fd11e8a034c None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:50.277799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02545f2b-ebc3-45fd-8494-3fd11e8a034c None admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:50.278808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02545f2b-ebc3-45fd-8494-3fd11e8a034c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:50.279120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02545f2b-ebc3-45fd-8494-3fd11e8a034c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:50.280556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02545f2b-ebc3-45fd-8494-3fd11e8a034c None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:50.292781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:50.305390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 632/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jun 29 08:25:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:50.312171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984ae2a-7206-4fd7-b8a3-e19f507c542f None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:50.439363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984ae2a-7206-4fd7-b8a3-e19f507c542f None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:50.482443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96b759-8df2-4ea8-9484-5ca6d3b39c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:50.483282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96b759-8df2-4ea8-9484-5ca6d3b39c2a None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:50.483559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96b759-8df2-4ea8-9484-5ca6d3b39c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:50.483829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96b759-8df2-4ea8-9484-5ca6d3b39c2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:50.497137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96b759-8df2-4ea8-9484-5ca6d3b39c2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:50.498798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96b759-8df2-4ea8-9484-5ca6d3b39c2a None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-fc96b759-8df2-4ea8-9484-5ca6d3b39c2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:50.519313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:50.520366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:50.521742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:50.521742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:50.521742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:50.521742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:50.521742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 637/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:50.527778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02545f2b-ebc3-45fd-8494-3fd11e8a034c None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:50.652643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02545f2b-ebc3-45fd-8494-3fd11e8a034c None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:50.689366 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bb553-0a14-4fe7-ba3f-3909fb5f3ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:50.690042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bb553-0a14-4fe7-ba3f-3909fb5f3ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:50.690301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bb553-0a14-4fe7-ba3f-3909fb5f3ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:50.690574 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bb553-0a14-4fe7-ba3f-3909fb5f3ae7 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:50.691364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58bb553-0a14-4fe7-ba3f-3909fb5f3ae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:50.691671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58bb553-0a14-4fe7-ba3f-3909fb5f3ae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:50.693056 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58bb553-0a14-4fe7-ba3f-3909fb5f3ae7 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:50.704986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 633/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:50.711161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96b759-8df2-4ea8-9484-5ca6d3b39c2a None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:50.748021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:50.824155 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96b759-8df2-4ea8-9484-5ca6d3b39c2a None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:50.863737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4c33a-cbae-4f3a-998b-cd4e495bdd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:50.864532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4c33a-cbae-4f3a-998b-cd4e495bdd5b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:50.864814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4c33a-cbae-4f3a-998b-cd4e495bdd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:50.865191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4c33a-cbae-4f3a-998b-cd4e495bdd5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:50.872948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4c33a-cbae-4f3a-998b-cd4e495bdd5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:50.874407 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4c33a-cbae-4f3a-998b-cd4e495bdd5b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-2da4c33a-cbae-4f3a-998b-cd4e495bdd5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:25:50.888918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:50.889983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:50.890933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:25:50.890933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:25:50.890933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:25:50.890933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:25:50.890933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:25:50.890933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:25:50.890933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:50.890933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:50.890933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 638/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:25:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:50.896458 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bb553-0a14-4fe7-ba3f-3909fb5f3ae7 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:50.970963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:51.025527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bb553-0a14-4fe7-ba3f-3909fb5f3ae7 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:51.062152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd51dd-3e3a-4967-b05c-c251f0678848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:51.062828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd51dd-3e3a-4967-b05c-c251f0678848 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:51.063110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd51dd-3e3a-4967-b05c-c251f0678848 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:51.063376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd51dd-3e3a-4967-b05c-c251f0678848 None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:51.064157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd51dd-3e3a-4967-b05c-c251f0678848 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:51.064461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd51dd-3e3a-4967-b05c-c251f0678848 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:51.065895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd51dd-3e3a-4967-b05c-c251f0678848 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:51.078822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 634/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:25:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:51.085265 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4c33a-cbae-4f3a-998b-cd4e495bdd5b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:51.178483 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:51.188665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4c33a-cbae-4f3a-998b-cd4e495bdd5b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:51.231130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09b78d-4964-48f1-b62c-38b22604dbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:51.232364 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09b78d-4964-48f1-b62c-38b22604dbe4 None admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:51.232736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09b78d-4964-48f1-b62c-38b22604dbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:51.233049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09b78d-4964-48f1-b62c-38b22604dbe4 None admin] PATH_INFO: `/v3/projects/c7a300f406e1459583100a1086caee9f/users/00a08f31638d4eaab5ca83d6f493db6c/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:51.235812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09b78d-4964-48f1-b62c-38b22604dbe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c7a300f406e1459583100a1086caee9f, user_id=00a08f31638d4eaab5ca83d6f493db6c, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:51.237647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09b78d-4964-48f1-b62c-38b22604dbe4 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:51.254181 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 639/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Wed Jun 29 08:25:51 2022] PUT /identity/v3/projects/c7a300f406e1459583100a1086caee9f/users/00a08f31638d4eaab5ca83d6f493db6c/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:25:53.272823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125f72a-9dc9-49e1-957b-d9c872a70a38 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:53.273435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125f72a-9dc9-49e1-957b-d9c872a70a38 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:53.273435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125f72a-9dc9-49e1-957b-d9c872a70a38 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:53.274722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 635/1274] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:25:53.281242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5fedf-f08a-496f-805f-f29d4950c712 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:53.281955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5fedf-f08a-496f-805f-f29d4950c712 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:53.282102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5fedf-f08a-496f-805f-f29d4950c712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:53.308236 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e8d5fedf-f08a-496f-805f-f29d4950c712 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:25:53.309436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e8d5fedf-f08a-496f-805f-f29d4950c712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:25:53.379582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 640/1275] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:25:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:25:53.410072 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125f72a-9dc9-49e1-957b-d9c872a70a38 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:53.528162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125f72a-9dc9-49e1-957b-d9c872a70a38 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:53.546277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:53.566471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c525e-9fe0-4e0b-be57-9dac62b3a394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:53.567303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c525e-9fe0-4e0b-be57-9dac62b3a394 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:53.567582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c525e-9fe0-4e0b-be57-9dac62b3a394 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:53.567857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c525e-9fe0-4e0b-be57-9dac62b3a394 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:53.578587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c525e-9fe0-4e0b-be57-9dac62b3a394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-033c525e-9fe0-4e0b-be57-9dac62b3a394 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 08:25:53.586114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:53.587286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:53.588214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 636/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Wed Jun 29 08:25:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:53.594126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5fedf-f08a-496f-805f-f29d4950c712 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:53.695173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:53.716176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d5fedf-f08a-496f-805f-f29d4950c712 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:53.752136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5237354-d1aa-45fa-886b-d32cef94b9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:53.752799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5237354-d1aa-45fa-886b-d32cef94b9eb None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:53.753074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5237354-d1aa-45fa-886b-d32cef94b9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:53.753348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5237354-d1aa-45fa-886b-d32cef94b9eb None admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:53.754092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5237354-d1aa-45fa-886b-d32cef94b9eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:53.754400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5237354-d1aa-45fa-886b-d32cef94b9eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:53.755946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5237354-d1aa-45fa-886b-d32cef94b9eb None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:53.764098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 641/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Wed Jun 29 08:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:25:53.773763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c525e-9fe0-4e0b-be57-9dac62b3a394 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:53.883785 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c525e-9fe0-4e0b-be57-9dac62b3a394 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:53.918092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42229fa-9661-4799-a0a8-d5141711caf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:53.918893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42229fa-9661-4799-a0a8-d5141711caf3 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:53.919171 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42229fa-9661-4799-a0a8-d5141711caf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:53.919444 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42229fa-9661-4799-a0a8-d5141711caf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:53.930616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42229fa-9661-4799-a0a8-d5141711caf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:53.932095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42229fa-9661-4799-a0a8-d5141711caf3 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-d42229fa-9661-4799-a0a8-d5141711caf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:53.949302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:53.950336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:53.951455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:53.951455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:53.951455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:53.951455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:53.951455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 637/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:25:53.956932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5237354-d1aa-45fa-886b-d32cef94b9eb None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:54.075233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5237354-d1aa-45fa-886b-d32cef94b9eb None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:54.120780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17309c79-edf6-47fd-99b5-f1772b279697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:54.121430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309c79-edf6-47fd-99b5-f1772b279697 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:54.121703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309c79-edf6-47fd-99b5-f1772b279697 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:54.121970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17309c79-edf6-47fd-99b5-f1772b279697 None admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:54.122721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17309c79-edf6-47fd-99b5-f1772b279697 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:54.123024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17309c79-edf6-47fd-99b5-f1772b279697 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:54.125104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17309c79-edf6-47fd-99b5-f1772b279697 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:54.139657 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 642/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:54.147058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42229fa-9661-4799-a0a8-d5141711caf3 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:54.182895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:54.261177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42229fa-9661-4799-a0a8-d5141711caf3 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:54.296729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f5c66-dbf4-4866-afbc-0790cc06a1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:54.297529 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f5c66-dbf4-4866-afbc-0790cc06a1cf None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:54.297776 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f5c66-dbf4-4866-afbc-0790cc06a1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:54.298066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f5c66-dbf4-4866-afbc-0790cc06a1cf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:54.305451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f5c66-dbf4-4866-afbc-0790cc06a1cf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:54.306915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f5c66-dbf4-4866-afbc-0790cc06a1cf None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-5a7f5c66-dbf4-4866-afbc-0790cc06a1cf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 08:25:54.320986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:54.322051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:54.323278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:54.323278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:54.323278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 08:25:54.323278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 08:25:54.323278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 08:25:54.323278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 08:25:54.323278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 08:25:54.323278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 08:25:54.323278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 08:25:54.323278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:54.323278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 638/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:54.328244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17309c79-edf6-47fd-99b5-f1772b279697 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:54.334196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:54.439865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17309c79-edf6-47fd-99b5-f1772b279697 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:54.482178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231f16e-0326-48a4-be22-5289a462de92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:54.482900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f16e-0326-48a4-be22-5289a462de92 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:54.483168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f16e-0326-48a4-be22-5289a462de92 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:54.483439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231f16e-0326-48a4-be22-5289a462de92 None admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:54.484284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231f16e-0326-48a4-be22-5289a462de92 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:54.484592 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231f16e-0326-48a4-be22-5289a462de92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:54.486037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231f16e-0326-48a4-be22-5289a462de92 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:54.503514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 643/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Wed Jun 29 08:25:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:54.511988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f5c66-dbf4-4866-afbc-0790cc06a1cf None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:54.607089 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:54.617878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f5c66-dbf4-4866-afbc-0790cc06a1cf None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:54.650650 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d140922-a20e-4559-a283-2d6814fb5e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:54.651317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d140922-a20e-4559-a283-2d6814fb5e36 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:54.651600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d140922-a20e-4559-a283-2d6814fb5e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:54.651879 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d140922-a20e-4559-a283-2d6814fb5e36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:54.720356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d140922-a20e-4559-a283-2d6814fb5e36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:54.721731 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d140922-a20e-4559-a283-2d6814fb5e36 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:54.811411 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:54.824654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 639/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Wed Jun 29 08:25:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 08:25:54.833492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231f16e-0326-48a4-be22-5289a462de92 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:54.896404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:54.937585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231f16e-0326-48a4-be22-5289a462de92 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:54.973487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d507f-8182-4fc7-95e3-a6cccd36f60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:54.974206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d507f-8182-4fc7-95e3-a6cccd36f60b None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:54.974581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d507f-8182-4fc7-95e3-a6cccd36f60b None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:54.974875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d507f-8182-4fc7-95e3-a6cccd36f60b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:54.985570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d507f-8182-4fc7-95e3-a6cccd36f60b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:54.987051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d507f-8182-4fc7-95e3-a6cccd36f60b None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-ed7d507f-8182-4fc7-95e3-a6cccd36f60b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 08:25:55.004676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:55.005709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:55.006750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 08:25:55.006750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 08:25:55.006750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 08:25:55.006750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:55.006750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 644/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 08:25:55.014443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d140922-a20e-4559-a283-2d6814fb5e36 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:55.125575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d140922-a20e-4559-a283-2d6814fb5e36 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:55.163751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a35c6-1d47-4032-94a7-d1d441c31e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:55.164408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a35c6-1d47-4032-94a7-d1d441c31e78 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:55.164690 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a35c6-1d47-4032-94a7-d1d441c31e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:55.164963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a35c6-1d47-4032-94a7-d1d441c31e78 None admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:55.165684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a35c6-1d47-4032-94a7-d1d441c31e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:55.165989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a35c6-1d47-4032-94a7-d1d441c31e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:55.167384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a35c6-1d47-4032-94a7-d1d441c31e78 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:55.178528 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 640/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Wed Jun 29 08:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:55.184993 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d507f-8182-4fc7-95e3-a6cccd36f60b None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:55.302022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d507f-8182-4fc7-95e3-a6cccd36f60b None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:55.318592 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:55.351756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847325d2-c1d2-4d4d-877a-f7ea377fb522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:55.353006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847325d2-c1d2-4d4d-877a-f7ea377fb522 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:55.353398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847325d2-c1d2-4d4d-877a-f7ea377fb522 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:55.353780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847325d2-c1d2-4d4d-877a-f7ea377fb522 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:55.365372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847325d2-c1d2-4d4d-877a-f7ea377fb522 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:55.366860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847325d2-c1d2-4d4d-877a-f7ea377fb522 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application [None req-847325d2-c1d2-4d4d-877a-f7ea377fb522 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 08:25:55.380883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:55.381861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:55.382874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 08:25:55.382874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 08:25:55.382874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 08:25:55.382874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 08:25:55.382874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 08:25:55.382874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 08:25:55.382874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 08:25:55.382874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: ERROR keystone.server.flask.application Jun 29 08:25:55.382874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 645/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Wed Jun 29 08:25:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:55.388313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a35c6-1d47-4032-94a7-d1d441c31e78 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:55.444236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:55.499491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a35c6-1d47-4032-94a7-d1d441c31e78 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:55.537487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d854ca7-df98-42b1-9109-43aee127dfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:55.538167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d854ca7-df98-42b1-9109-43aee127dfbc None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:55.538447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d854ca7-df98-42b1-9109-43aee127dfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:55.538738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d854ca7-df98-42b1-9109-43aee127dfbc None admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:55.539540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d854ca7-df98-42b1-9109-43aee127dfbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:55.539843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d854ca7-df98-42b1-9109-43aee127dfbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:55.541274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d854ca7-df98-42b1-9109-43aee127dfbc None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:55.554385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 641/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Wed Jun 29 08:25:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:55.560842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847325d2-c1d2-4d4d-877a-f7ea377fb522 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:55.670963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847325d2-c1d2-4d4d-877a-f7ea377fb522 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:55.705815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9703b41e-820e-425c-a8e0-7c70022d8c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:55.706447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703b41e-820e-425c-a8e0-7c70022d8c55 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:55.706743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703b41e-820e-425c-a8e0-7c70022d8c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:55.707016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9703b41e-820e-425c-a8e0-7c70022d8c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:55.707835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703b41e-820e-425c-a8e0-7c70022d8c55 None admin] RBAC: Adding query filter params (role.id=d7286245c7bf41c9b699f41f0a2fa69e, scope.project.id=c7a300f406e1459583100a1086caee9f, user.id=00a08f31638d4eaab5ca83d6f493db6c) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:55.708148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703b41e-820e-425c-a8e0-7c70022d8c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:55.709684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9703b41e-820e-425c-a8e0-7c70022d8c55 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:55.715945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 646/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Wed Jun 29 08:25:55 2022] GET /identity/v3/role_assignments?user.id=00a08f31638d4eaab5ca83d6f493db6c&scope.project.id=c7a300f406e1459583100a1086caee9f&role.id=d7286245c7bf41c9b699f41f0a2fa69e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:57.755881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ead9aa-c3d8-450d-b1fd-f95374567a5f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:57.756755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ead9aa-c3d8-450d-b1fd-f95374567a5f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:57.756863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ead9aa-c3d8-450d-b1fd-f95374567a5f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:57.759081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 642/1288] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:25:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:25:57.765517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec0194-b4ac-4108-bd49-006392b7156f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:57.765951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec0194-b4ac-4108-bd49-006392b7156f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:57.766088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec0194-b4ac-4108-bd49-006392b7156f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:57.786061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d9ec0194-b4ac-4108-bd49-006392b7156f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:25:57.787812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d9ec0194-b4ac-4108-bd49-006392b7156f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:25:57.834127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:57.877807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 647/1289] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:25:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:25:57.901548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ead9aa-c3d8-450d-b1fd-f95374567a5f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:58.017184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ead9aa-c3d8-450d-b1fd-f95374567a5f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:58.027930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:25:58.057365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092a35d-fb3b-4df5-891f-631c1bf53a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:58.058168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092a35d-fb3b-4df5-891f-631c1bf53a9e None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:58.058454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092a35d-fb3b-4df5-891f-631c1bf53a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:58.058725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092a35d-fb3b-4df5-891f-631c1bf53a9e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:58.059416 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092a35d-fb3b-4df5-891f-631c1bf53a9e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:58.060803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092a35d-fb3b-4df5-891f-631c1bf53a9e None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-3092a35d-fb3b-4df5-891f-631c1bf53a9e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:25:58.073244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:25:58.074593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:25:58.075696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 643/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jun 29 08:25:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:25:58.083122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec0194-b4ac-4108-bd49-006392b7156f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:58.188532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec0194-b4ac-4108-bd49-006392b7156f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:58.227136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b95cf-925a-4400-944c-353d0da9167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:58.227848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b95cf-925a-4400-944c-353d0da9167c None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:58.228139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b95cf-925a-4400-944c-353d0da9167c None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:58.228447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b95cf-925a-4400-944c-353d0da9167c None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:58.229258 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b95cf-925a-4400-944c-353d0da9167c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:58.229602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b95cf-925a-4400-944c-353d0da9167c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:58.231200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b95cf-925a-4400-944c-353d0da9167c None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:58.237093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 648/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Jun 29 08:25:58 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:25:58.244091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092a35d-fb3b-4df5-891f-631c1bf53a9e None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:25:58.349675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092a35d-fb3b-4df5-891f-631c1bf53a9e None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:25:58.383893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854b201-1457-4c33-bb1b-2d0259d5c37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:25:58.384542 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854b201-1457-4c33-bb1b-2d0259d5c37b None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:25:58.384822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854b201-1457-4c33-bb1b-2d0259d5c37b None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:25:58.385066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854b201-1457-4c33-bb1b-2d0259d5c37b None admin] PATH_INFO: `/v3/services` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:25:58.385802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854b201-1457-4c33-bb1b-2d0259d5c37b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:25:58.386103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854b201-1457-4c33-bb1b-2d0259d5c37b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:25:58.387505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854b201-1457-4c33-bb1b-2d0259d5c37b None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:25:58.392114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 644/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Jun 29 08:25:58 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:26:00.370599 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3095b-81ec-4f0c-8e91-0f5deeeef4c4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:00.371270 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3095b-81ec-4f0c-8e91-0f5deeeef4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:00.371270 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c3095b-81ec-4f0c-8e91-0f5deeeef4c4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:00.372550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 649/1293] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:26:00.378849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa221f4a-77be-4ec7-a0d9-d312c6d6e35e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:00.379616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa221f4a-77be-4ec7-a0d9-d312c6d6e35e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:00.379774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa221f4a-77be-4ec7-a0d9-d312c6d6e35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:00.397437 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-aa221f4a-77be-4ec7-a0d9-d312c6d6e35e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:26:00.398628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-aa221f4a-77be-4ec7-a0d9-d312c6d6e35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:26:00.429161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:26:00.476448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 645/1294] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:26:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 08:26:00.500291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3095b-81ec-4f0c-8e91-0f5deeeef4c4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:26:00.539518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:26:00.617721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c3095b-81ec-4f0c-8e91-0f5deeeef4c4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:26:00.652839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57556a9e-3907-44c3-86f7-c0a82e48eb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:26:00.653420 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57556a9e-3907-44c3-86f7-c0a82e48eb00 None admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:00.653706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57556a9e-3907-44c3-86f7-c0a82e48eb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:00.653978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57556a9e-3907-44c3-86f7-c0a82e48eb00 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:00.654722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57556a9e-3907-44c3-86f7-c0a82e48eb00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:26:00.656098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57556a9e-3907-44c3-86f7-c0a82e48eb00 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:26:00.672453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 650/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Wed Jun 29 08:26:00 2022] POST /identity/v3/services => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:26:02.703495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69493d1a-54b6-4593-adf8-57fb50f642f9 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:02.704393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69493d1a-54b6-4593-adf8-57fb50f642f9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:02.704393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69493d1a-54b6-4593-adf8-57fb50f642f9 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:02.706569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 646/1296] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:26:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:26:02.715011 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b3dcf-35d2-4db4-8db5-eec4334b98a1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:02.715459 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b3dcf-35d2-4db4-8db5-eec4334b98a1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:02.715459 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b3dcf-35d2-4db4-8db5-eec4334b98a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:02.733473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a32b3dcf-35d2-4db4-8db5-eec4334b98a1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:26:02.734640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a32b3dcf-35d2-4db4-8db5-eec4334b98a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:26:02.809548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 651/1297] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:26:02 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:26:02.833072 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69493d1a-54b6-4593-adf8-57fb50f642f9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:26:02.945320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69493d1a-54b6-4593-adf8-57fb50f642f9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:26:02.982038 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bdc16-ecf0-4ccc-930a-9066764493c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:26:02.982753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bdc16-ecf0-4ccc-930a-9066764493c4 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:02.983081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bdc16-ecf0-4ccc-930a-9066764493c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:02.983353 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bdc16-ecf0-4ccc-930a-9066764493c4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:02.984011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bdc16-ecf0-4ccc-930a-9066764493c4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:26:02.985398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bdc16-ecf0-4ccc-930a-9066764493c4 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-279bdc16-ecf0-4ccc-930a-9066764493c4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:26:02.997404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:26:02.998522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:26:02.999500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 647/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jun 29 08:26:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:26:03.004647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b3dcf-35d2-4db4-8db5-eec4334b98a1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:26:03.024894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:26:03.111515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b3dcf-35d2-4db4-8db5-eec4334b98a1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:26:03.147146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9423e-9a7c-4704-9e4a-563113bbc436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:26:03.147803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9423e-9a7c-4704-9e4a-563113bbc436 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:03.148082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9423e-9a7c-4704-9e4a-563113bbc436 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:03.148351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9423e-9a7c-4704-9e4a-563113bbc436 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:03.149055 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9423e-9a7c-4704-9e4a-563113bbc436 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:26:03.149360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9423e-9a7c-4704-9e4a-563113bbc436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:26:03.150738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9423e-9a7c-4704-9e4a-563113bbc436 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:26:03.155983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 652/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Jun 29 08:26:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:26:03.162193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bdc16-ecf0-4ccc-930a-9066764493c4 None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:26:03.195298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:26:03.270180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bdc16-ecf0-4ccc-930a-9066764493c4 None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:26:03.304740 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc8f9e-c79b-4faa-8110-4bfec1415553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:26:03.305390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc8f9e-c79b-4faa-8110-4bfec1415553 None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:03.305666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc8f9e-c79b-4faa-8110-4bfec1415553 None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:03.305938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc8f9e-c79b-4faa-8110-4bfec1415553 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:03.306732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc8f9e-c79b-4faa-8110-4bfec1415553 None admin] RBAC: Adding query filter params (interface=public, service_id=5fe40105fc0647ca84d87db4bb153cc6, region_id=RegionOne) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:26:03.307055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc8f9e-c79b-4faa-8110-4bfec1415553 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:26:03.308409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc8f9e-c79b-4faa-8110-4bfec1415553 None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:26:03.313544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 648/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Wed Jun 29 08:26:03 2022] GET /identity/v3/endpoints?service_id=5fe40105fc0647ca84d87db4bb153cc6&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:26:03.320396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9423e-9a7c-4704-9e4a-563113bbc436 None admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:26:03.423181 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:26:03.442469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9423e-9a7c-4704-9e4a-563113bbc436 None admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:26:03.479408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163a8a3-05c9-4e44-b093-8175ae39e400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:26:03.480085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163a8a3-05c9-4e44-b093-8175ae39e400 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:03.480367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163a8a3-05c9-4e44-b093-8175ae39e400 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:03.480640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163a8a3-05c9-4e44-b093-8175ae39e400 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:03.481333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163a8a3-05c9-4e44-b093-8175ae39e400 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:26:03.482722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163a8a3-05c9-4e44-b093-8175ae39e400 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:26:03.489531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 653/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Wed Jun 29 08:26:03 2022] GET /identity/v3/services => generated 2191 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:26:05.480323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccf281-ccef-46ec-80a3-ea84ed864c5b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:05.480816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccf281-ccef-46ec-80a3-ea84ed864c5b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:05.480928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccf281-ccef-46ec-80a3-ea84ed864c5b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:05.483118 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 649/1302] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:26:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:26:05.490721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d1a9a-1a89-410e-bde8-8b66740eb170 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:05.491191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d1a9a-1a89-410e-bde8-8b66740eb170 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:05.491284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d1a9a-1a89-410e-bde8-8b66740eb170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:05.509576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0c9d1a9a-1a89-410e-bde8-8b66740eb170 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:26:05.510750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c9d1a9a-1a89-410e-bde8-8b66740eb170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:26:05.578547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 654/1303] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:26:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 08:26:05.609834 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccf281-ccef-46ec-80a3-ea84ed864c5b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:26:05.729496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:26:05.736798 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccf281-ccef-46ec-80a3-ea84ed864c5b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:26:05.771870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75716166-a77e-4aad-b51e-733cbfa10cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:26:05.772657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75716166-a77e-4aad-b51e-733cbfa10cac None admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:05.772935 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75716166-a77e-4aad-b51e-733cbfa10cac None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:05.773207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75716166-a77e-4aad-b51e-733cbfa10cac None admin] PATH_INFO: `/v3/services/placement` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:05.773857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75716166-a77e-4aad-b51e-733cbfa10cac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:26:05.775262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75716166-a77e-4aad-b51e-733cbfa10cac None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application [None req-75716166-a77e-4aad-b51e-733cbfa10cac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 08:26:05.787348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 08:26:05.788462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application with Lock( Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self._enter() Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application return self.creator() Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application created_value = creator( Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: ERROR keystone.server.flask.application Jun 29 08:26:05.792287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 650/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Wed Jun 29 08:26:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 08:26:05.797199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d1a9a-1a89-410e-bde8-8b66740eb170 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:26:05.883153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:26:05.910524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d1a9a-1a89-410e-bde8-8b66740eb170 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:26:05.948358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aafdfd-ff50-4e80-8415-8043585471c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:26:05.949033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aafdfd-ff50-4e80-8415-8043585471c2 None admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:05.949319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aafdfd-ff50-4e80-8415-8043585471c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:05.949594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aafdfd-ff50-4e80-8415-8043585471c2 None admin] PATH_INFO: `/v3/services` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:05.950312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aafdfd-ff50-4e80-8415-8043585471c2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:26:05.950611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aafdfd-ff50-4e80-8415-8043585471c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:26:05.952042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aafdfd-ff50-4e80-8415-8043585471c2 None admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:26:05.957852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 655/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Wed Jun 29 08:26:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:26:05.967143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75716166-a77e-4aad-b51e-733cbfa10cac None admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:26:06.072296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75716166-a77e-4aad-b51e-733cbfa10cac None admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:26:06.108317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e2766-f2b6-4a2b-8151-439da2efaaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:26:06.108971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e2766-f2b6-4a2b-8151-439da2efaaeb None admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:26:06.109260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e2766-f2b6-4a2b-8151-439da2efaaeb None admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:26:06.109536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e2766-f2b6-4a2b-8151-439da2efaaeb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:26:06.110279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e2766-f2b6-4a2b-8151-439da2efaaeb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:26:06.111684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e2766-f2b6-4a2b-8151-439da2efaaeb None admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:26:06.134438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 651/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Wed Jun 29 08:26:05 2022] POST /identity/v3/endpoints => generated 331 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:26:15.954269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=30 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:26:16.129531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:09.122167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5e83-d417-4565-be6c-1c07b5423601 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:09.123085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5e83-d417-4565-be6c-1c07b5423601 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:09.123522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5e83-d417-4565-be6c-1c07b5423601 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:09.125078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 656/1307] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:09.130545 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856d509-004f-452a-b593-80908a1d14c8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:09.131004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856d509-004f-452a-b593-80908a1d14c8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:09.131528 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856d509-004f-452a-b593-80908a1d14c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:09.264355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0856d509-004f-452a-b593-80908a1d14c8 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:09.265827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0856d509-004f-452a-b593-80908a1d14c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:09.468978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 652/1308] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:27:09.523444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e69ebd-0492-4938-8439-7b4ab3e9d274 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:09.523816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e69ebd-0492-4938-8439-7b4ab3e9d274 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:09.524001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e69ebd-0492-4938-8439-7b4ab3e9d274 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:09.525391 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 657/1309] 10.222.0.27 () {54 vars in 964 bytes} [Wed Jun 29 08:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:09.532035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67c674-e55e-48b4-9fcf-112f67afe713 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:09.532327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67c674-e55e-48b4-9fcf-112f67afe713 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:09.532607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67c674-e55e-48b4-9fcf-112f67afe713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:09.659835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ca67c674-e55e-48b4-9fcf-112f67afe713 None None] MFA Rules not processed for user `49a1439029fa4b719bc114c3f59e0050`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:09.661060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ca67c674-e55e-48b4-9fcf-112f67afe713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:09.684228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:09.746762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 653/1310] 10.222.0.27 () {58 vars in 1061 bytes} [Wed Jun 29 08:27:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:27:09.755955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e69ebd-0492-4938-8439-7b4ab3e9d274 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:09.758091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-65e69ebd-0492-4938-8439-7b4ab3e9d274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:09.894302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e69ebd-0492-4938-8439-7b4ab3e9d274 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:09.983901 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83338528-d0ce-4c1a-af87-28d4191e8e03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:09.984820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83338528-d0ce-4c1a-af87-28d4191e8e03 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:09.985435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83338528-d0ce-4c1a-af87-28d4191e8e03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:09.986245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83338528-d0ce-4c1a-af87-28d4191e8e03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:09.988066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-83338528-d0ce-4c1a-af87-28d4191e8e03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:10.049811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:10.111386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83338528-d0ce-4c1a-af87-28d4191e8e03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:10.113063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83338528-d0ce-4c1a-af87-28d4191e8e03 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:10.133659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 658/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:12.503147 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b39f4b-81c2-445c-b6dc-5edd2b501a93 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:12.503706 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b39f4b-81c2-445c-b6dc-5edd2b501a93 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:12.503802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b39f4b-81c2-445c-b6dc-5edd2b501a93 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:12.505432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 654/1312] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:12.510937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a917fcf-8e93-401c-ae68-cbb8972e6751 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:12.511357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a917fcf-8e93-401c-ae68-cbb8972e6751 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:12.511584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a917fcf-8e93-401c-ae68-cbb8972e6751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:12.593966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3a917fcf-8e93-401c-ae68-cbb8972e6751 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:12.595214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3a917fcf-8e93-401c-ae68-cbb8972e6751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:12.694136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 659/1313] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:12.745282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a69a4-ec81-4e43-8385-e1b112a2f451 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:12.745575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a69a4-ec81-4e43-8385-e1b112a2f451 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:12.745800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a69a4-ec81-4e43-8385-e1b112a2f451 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:12.747261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 655/1314] 10.222.0.27 () {54 vars in 964 bytes} [Wed Jun 29 08:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:12.754338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8a87f-bbaf-443a-958c-b76e7e579a58 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:12.754629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8a87f-bbaf-443a-958c-b76e7e579a58 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:12.755045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8a87f-bbaf-443a-958c-b76e7e579a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:12.765989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:12.832021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-31f8a87f-bbaf-443a-958c-b76e7e579a58 None None] MFA Rules not processed for user `49a1439029fa4b719bc114c3f59e0050`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:12.833271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-31f8a87f-bbaf-443a-958c-b76e7e579a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:12.922317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 660/1315] 10.222.0.27 () {58 vars in 1061 bytes} [Wed Jun 29 08:27:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:12.931737 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a69a4-ec81-4e43-8385-e1b112a2f451 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:12.933310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-726a69a4-ec81-4e43-8385-e1b112a2f451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:13.063001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a69a4-ec81-4e43-8385-e1b112a2f451 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:13.150420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:13.153054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c1ee5-61a4-44a0-951d-2e5ce780cab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:13.153813 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c1ee5-61a4-44a0-951d-2e5ce780cab5 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:13.154100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c1ee5-61a4-44a0-951d-2e5ce780cab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:13.154465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c1ee5-61a4-44a0-951d-2e5ce780cab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:13.156584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5c6c1ee5-61a4-44a0-951d-2e5ce780cab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:13.271382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c1ee5-61a4-44a0-951d-2e5ce780cab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:13.272869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c1ee5-61a4-44a0-951d-2e5ce780cab5 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:13.294648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 656/1316] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:15.880514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eb2ff-59cb-45e0-8f8a-0357d2f2d3ae None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:15.881083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eb2ff-59cb-45e0-8f8a-0357d2f2d3ae None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:15.881376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2eb2ff-59cb-45e0-8f8a-0357d2f2d3ae None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:15.882828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 661/1317] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:15.889711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9dced-d279-4248-bcff-bc5a39cb33ce None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:15.890482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9dced-d279-4248-bcff-bc5a39cb33ce None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:15.890608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9dced-d279-4248-bcff-bc5a39cb33ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:15.963408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-dfa9dced-d279-4248-bcff-bc5a39cb33ce None None] MFA Rules not processed for user `c2a06e0093704ec0865c012a4c2427dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:15.964967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dfa9dced-d279-4248-bcff-bc5a39cb33ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:16.021497 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:16.076263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 657/1318] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:16.119392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2eb2ff-59cb-45e0-8f8a-0357d2f2d3ae None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:16.120927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1f2eb2ff-59cb-45e0-8f8a-0357d2f2d3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:16.230816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2eb2ff-59cb-45e0-8f8a-0357d2f2d3ae None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:16.233586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:16.312115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6d0f1d-0341-442f-9bc7-f7f9a1610385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:16.312759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d0f1d-0341-442f-9bc7-f7f9a1610385 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:16.313037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d0f1d-0341-442f-9bc7-f7f9a1610385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:16.313315 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6d0f1d-0341-442f-9bc7-f7f9a1610385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:16.314994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ce6d0f1d-0341-442f-9bc7-f7f9a1610385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:16.440430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d0f1d-0341-442f-9bc7-f7f9a1610385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:16.441878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6d0f1d-0341-442f-9bc7-f7f9a1610385 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:16.461349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 662/1319] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:18.999139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a93a3-8773-4ac0-8968-c1ac5cc3df28 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:18.999988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a93a3-8773-4ac0-8968-c1ac5cc3df28 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:18.999988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a93a3-8773-4ac0-8968-c1ac5cc3df28 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:19.002298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 658/1320] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:19.008632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94c07-98f9-43cc-a3c8-dce0c8a2b69e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:19.009084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94c07-98f9-43cc-a3c8-dce0c8a2b69e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:19.009196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c94c07-98f9-43cc-a3c8-dce0c8a2b69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:19.059330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:19.086565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-04c94c07-98f9-43cc-a3c8-dce0c8a2b69e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:19.087811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-04c94c07-98f9-43cc-a3c8-dce0c8a2b69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:19.197812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 663/1321] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:19.236290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a93a3-8773-4ac0-8968-c1ac5cc3df28 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:19.237911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-814a93a3-8773-4ac0-8968-c1ac5cc3df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:19.343218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a93a3-8773-4ac0-8968-c1ac5cc3df28 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:19.417650 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d5dbd-34aa-4335-b47f-4239d69de57c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:19.418303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d5dbd-34aa-4335-b47f-4239d69de57c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:19.418582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d5dbd-34aa-4335-b47f-4239d69de57c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:19.418889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d5dbd-34aa-4335-b47f-4239d69de57c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:19.420525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-322d5dbd-34aa-4335-b47f-4239d69de57c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:19.428467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:19.549831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d5dbd-34aa-4335-b47f-4239d69de57c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:19.551361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d5dbd-34aa-4335-b47f-4239d69de57c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:19.570055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 659/1322] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:22.263599 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cf503-58dd-4142-9412-caf3ae85bc1d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:22.264176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cf503-58dd-4142-9412-caf3ae85bc1d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:22.264176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cf503-58dd-4142-9412-caf3ae85bc1d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:22.265724 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 664/1323] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:22.271894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2190d-c23a-453b-a95c-76d7403291de None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:22.272322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2190d-c23a-453b-a95c-76d7403291de None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:22.272417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2190d-c23a-453b-a95c-76d7403291de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:22.347152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c3e2190d-c23a-453b-a95c-76d7403291de None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:22.348408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c3e2190d-c23a-453b-a95c-76d7403291de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:22.408554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:22.455253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 660/1324] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:22.492006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cf503-58dd-4142-9412-caf3ae85bc1d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:22.493717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-026cf503-58dd-4142-9412-caf3ae85bc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:22.617373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cf503-58dd-4142-9412-caf3ae85bc1d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:22.631114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:22.695927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393fad5-2479-46bf-96f6-f66b1d5eca7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:22.696589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393fad5-2479-46bf-96f6-f66b1d5eca7d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:22.696849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393fad5-2479-46bf-96f6-f66b1d5eca7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:22.697133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393fad5-2479-46bf-96f6-f66b1d5eca7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:22.698807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6393fad5-2479-46bf-96f6-f66b1d5eca7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:22.809857 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393fad5-2479-46bf-96f6-f66b1d5eca7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:22.811338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393fad5-2479-46bf-96f6-f66b1d5eca7d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:22.830159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 665/1325] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:24.918741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031b508-155e-4067-8f6a-9486745d1826 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:24.919274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031b508-155e-4067-8f6a-9486745d1826 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:24.919274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031b508-155e-4067-8f6a-9486745d1826 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:24.920668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 661/1326] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:24.925977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1d22-4c30-4814-9162-46b61162ac21 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:24.926404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1d22-4c30-4814-9162-46b61162ac21 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:24.926492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1d22-4c30-4814-9162-46b61162ac21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:24.975629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:24.995078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-248f1d22-4c30-4814-9162-46b61162ac21 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:24.996293 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-248f1d22-4c30-4814-9162-46b61162ac21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:25.096186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 666/1327] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:25.131917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031b508-155e-4067-8f6a-9486745d1826 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:25.133512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5031b508-155e-4067-8f6a-9486745d1826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:25.243263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031b508-155e-4067-8f6a-9486745d1826 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:25.320517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a9ec5-b22c-4577-a04c-1e2151980229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:25.321183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a9ec5-b22c-4577-a04c-1e2151980229 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:25.321462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a9ec5-b22c-4577-a04c-1e2151980229 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:25.322120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a9ec5-b22c-4577-a04c-1e2151980229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:25.323913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ee1a9ec5-b22c-4577-a04c-1e2151980229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:25.343884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:25.446776 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a9ec5-b22c-4577-a04c-1e2151980229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:25.448225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a9ec5-b22c-4577-a04c-1e2151980229 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:25.469899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 662/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:27.542915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d456-98d1-4309-bfb8-35d902bc246e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:27.543566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d456-98d1-4309-bfb8-35d902bc246e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:27.543566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2d456-98d1-4309-bfb8-35d902bc246e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:27.544859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 667/1329] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:27.550136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88230155-e2e5-4599-b772-2dc3ed73ffa6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:27.550613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88230155-e2e5-4599-b772-2dc3ed73ffa6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:27.550753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88230155-e2e5-4599-b772-2dc3ed73ffa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:27.621264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-88230155-e2e5-4599-b772-2dc3ed73ffa6 None None] MFA Rules not processed for user `c2a06e0093704ec0865c012a4c2427dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:27.622410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-88230155-e2e5-4599-b772-2dc3ed73ffa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:27.694476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:27.732379 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 663/1330] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:27.769852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2d456-98d1-4309-bfb8-35d902bc246e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:27.772355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c2b2d456-98d1-4309-bfb8-35d902bc246e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:27.888059 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2d456-98d1-4309-bfb8-35d902bc246e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:27.910108 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:27.972464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d9d44-2e7f-44f3-adbb-bc183ce9b9b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:27.973153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d9d44-2e7f-44f3-adbb-bc183ce9b9b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:27.973443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d9d44-2e7f-44f3-adbb-bc183ce9b9b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:27.974037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d9d44-2e7f-44f3-adbb-bc183ce9b9b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:27.976095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-497d9d44-2e7f-44f3-adbb-bc183ce9b9b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:28.105665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d9d44-2e7f-44f3-adbb-bc183ce9b9b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:28.107156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d9d44-2e7f-44f3-adbb-bc183ce9b9b5 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:28.129894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 668/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:30.397597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151277d-6c4e-4d58-b021-ca0aa87d4c78 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:30.398176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151277d-6c4e-4d58-b021-ca0aa87d4c78 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:30.398176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151277d-6c4e-4d58-b021-ca0aa87d4c78 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:30.399571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 664/1332] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:30.405989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045cbe3-49bb-4e03-a24d-a5544bba1891 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:30.406759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045cbe3-49bb-4e03-a24d-a5544bba1891 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:30.406913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045cbe3-49bb-4e03-a24d-a5544bba1891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:30.479667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:30.483030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a045cbe3-49bb-4e03-a24d-a5544bba1891 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:30.485314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a045cbe3-49bb-4e03-a24d-a5544bba1891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:30.593122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 669/1333] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:30.628686 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151277d-6c4e-4d58-b021-ca0aa87d4c78 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:30.630790 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5151277d-6c4e-4d58-b021-ca0aa87d4c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:30.743646 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151277d-6c4e-4d58-b021-ca0aa87d4c78 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:30.822901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6dfa8e-95fc-4cb7-995e-c82389181b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:30.823549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dfa8e-95fc-4cb7-995e-c82389181b1c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:30.823826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dfa8e-95fc-4cb7-995e-c82389181b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:30.824099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6dfa8e-95fc-4cb7-995e-c82389181b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:30.825739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4d6dfa8e-95fc-4cb7-995e-c82389181b1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:30.853496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:30.939399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6dfa8e-95fc-4cb7-995e-c82389181b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:30.940761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6dfa8e-95fc-4cb7-995e-c82389181b1c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:30.959907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 665/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:33.043029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60377e40-ebef-4294-81d3-cee10aa90667 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:33.043606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60377e40-ebef-4294-81d3-cee10aa90667 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:33.043606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60377e40-ebef-4294-81d3-cee10aa90667 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:33.044976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 670/1335] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:33.051288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b9b7c-4538-4829-adc7-6cf43bae14bd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:33.051856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b9b7c-4538-4829-adc7-6cf43bae14bd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:33.051999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b9b7c-4538-4829-adc7-6cf43bae14bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:33.120709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-859b9b7c-4538-4829-adc7-6cf43bae14bd None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:33.121874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-859b9b7c-4538-4829-adc7-6cf43bae14bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:33.204268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:33.230468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 666/1336] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:33.271324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60377e40-ebef-4294-81d3-cee10aa90667 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:33.273743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-60377e40-ebef-4294-81d3-cee10aa90667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:33.386352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60377e40-ebef-4294-81d3-cee10aa90667 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:33.416358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:33.465607 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095892a-e265-492d-85b2-d5a24fc6e132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:33.466268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095892a-e265-492d-85b2-d5a24fc6e132 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:33.466541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095892a-e265-492d-85b2-d5a24fc6e132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:33.466851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095892a-e265-492d-85b2-d5a24fc6e132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:33.468492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0095892a-e265-492d-85b2-d5a24fc6e132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:33.585955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095892a-e265-492d-85b2-d5a24fc6e132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:33.587421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095892a-e265-492d-85b2-d5a24fc6e132 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:33.607979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 671/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:36.273754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623db631-6c36-48da-93fb-bc11447d4bb4 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:36.274588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623db631-6c36-48da-93fb-bc11447d4bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:36.274803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623db631-6c36-48da-93fb-bc11447d4bb4 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:36.277091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 667/1338] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:36.284558 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05caaf-0b89-4725-b521-f36b8c36f3e3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:36.285002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05caaf-0b89-4725-b521-f36b8c36f3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:36.285092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05caaf-0b89-4725-b521-f36b8c36f3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:36.355022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bb05caaf-0b89-4725-b521-f36b8c36f3e3 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:36.356238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bb05caaf-0b89-4725-b521-f36b8c36f3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:36.365441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:36.465871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 672/1339] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:36.504851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623db631-6c36-48da-93fb-bc11447d4bb4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:36.506564 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-623db631-6c36-48da-93fb-bc11447d4bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:36.631329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623db631-6c36-48da-93fb-bc11447d4bb4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:36.719069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b100936-c075-415e-b87a-3ef24683a2f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:36.719724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b100936-c075-415e-b87a-3ef24683a2f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:36.720009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b100936-c075-415e-b87a-3ef24683a2f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:36.720258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b100936-c075-415e-b87a-3ef24683a2f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:36.721903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8b100936-c075-415e-b87a-3ef24683a2f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:36.760017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:36.838747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b100936-c075-415e-b87a-3ef24683a2f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:36.840552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b100936-c075-415e-b87a-3ef24683a2f1 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:36.859209 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 668/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:41.041882 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b2181-1b8f-4d6a-beea-68bbef30a3bb None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:41.042402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b2181-1b8f-4d6a-beea-68bbef30a3bb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:41.042402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b2181-1b8f-4d6a-beea-68bbef30a3bb None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:41.043753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 673/1341] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:41.049087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d45dc-40a9-43b1-b52f-7ec7446ae46d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:41.049577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d45dc-40a9-43b1-b52f-7ec7446ae46d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:41.049577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d45dc-40a9-43b1-b52f-7ec7446ae46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:41.126243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-481d45dc-40a9-43b1-b52f-7ec7446ae46d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:41.127483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-481d45dc-40a9-43b1-b52f-7ec7446ae46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:41.222464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:41.236510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 669/1342] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:41.276052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b2181-1b8f-4d6a-beea-68bbef30a3bb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:41.278330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a47b2181-1b8f-4d6a-beea-68bbef30a3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:41.374730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b2181-1b8f-4d6a-beea-68bbef30a3bb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:41.380853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce08d8-6f89-485b-b95a-421ccd8f2d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:41.380853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce08d8-6f89-485b-b95a-421ccd8f2d6e service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:41.380853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce08d8-6f89-485b-b95a-421ccd8f2d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:41.380853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce08d8-6f89-485b-b95a-421ccd8f2d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:41.390874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce08d8-6f89-485b-b95a-421ccd8f2d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:41.392306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce08d8-6f89-485b-b95a-421ccd8f2d6e service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:41.401938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 674/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:44.029155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c35af4-b47e-44d8-a7d1-8bfc931645ca None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:44.029754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c35af4-b47e-44d8-a7d1-8bfc931645ca None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:44.029754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c35af4-b47e-44d8-a7d1-8bfc931645ca None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:44.031111 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 670/1344] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:44.036430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9041509-cb3c-412d-80f3-c6cd468a94e2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:44.036888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9041509-cb3c-412d-80f3-c6cd468a94e2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:44.036982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9041509-cb3c-412d-80f3-c6cd468a94e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:44.069331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e9041509-cb3c-412d-80f3-c6cd468a94e2 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:44.070543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e9041509-cb3c-412d-80f3-c6cd468a94e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:44.101318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:44.153976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 675/1345] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:44.190964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c35af4-b47e-44d8-a7d1-8bfc931645ca None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:44.192794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e3c35af4-b47e-44d8-a7d1-8bfc931645ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:44.207341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c35af4-b47e-44d8-a7d1-8bfc931645ca None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:44.211570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf790ac-cbfd-4231-a5ab-9290df45b5e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:44.212272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf790ac-cbfd-4231-a5ab-9290df45b5e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:44.212533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf790ac-cbfd-4231-a5ab-9290df45b5e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:44.212832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf790ac-cbfd-4231-a5ab-9290df45b5e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:44.224606 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf790ac-cbfd-4231-a5ab-9290df45b5e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:44.226153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf790ac-cbfd-4231-a5ab-9290df45b5e4 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:44.237075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 671/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:48.481190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bcf31-3dd3-427c-b44e-d97203bbc15e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:48.481733 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bcf31-3dd3-427c-b44e-d97203bbc15e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:48.481733 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bcf31-3dd3-427c-b44e-d97203bbc15e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:48.483213 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 676/1347] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:48.488585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192c6b4-1c6c-4041-9ae7-8a93e0882787 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:48.489128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192c6b4-1c6c-4041-9ae7-8a93e0882787 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:48.489225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192c6b4-1c6c-4041-9ae7-8a93e0882787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:48.508665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4192c6b4-1c6c-4041-9ae7-8a93e0882787 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:48.510172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4192c6b4-1c6c-4041-9ae7-8a93e0882787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:48.518769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 672/1348] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:48.561021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766bcf31-3dd3-427c-b44e-d97203bbc15e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:27:48.572937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766bcf31-3dd3-427c-b44e-d97203bbc15e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:27:48.576868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710257fa-a6a2-44e9-a188-0668ca8de674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:27:48.577500 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710257fa-a6a2-44e9-a188-0668ca8de674 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:48.577785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710257fa-a6a2-44e9-a188-0668ca8de674 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:48.578062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710257fa-a6a2-44e9-a188-0668ca8de674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:48.589338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710257fa-a6a2-44e9-a188-0668ca8de674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:27:48.590857 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710257fa-a6a2-44e9-a188-0668ca8de674 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:27:48.601175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 677/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:27:53.768106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c94b06-a65b-4471-aca6-b784949319c3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:53.769158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c94b06-a65b-4471-aca6-b784949319c3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:53.769668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c94b06-a65b-4471-aca6-b784949319c3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:53.772313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 673/1350] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:27:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:53.777828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba32033-bc9d-4029-9b73-e6cb323611c3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:53.778673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba32033-bc9d-4029-9b73-e6cb323611c3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:53.779350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba32033-bc9d-4029-9b73-e6cb323611c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:53.857264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6ba32033-bc9d-4029-9b73-e6cb323611c3 None None] MFA Rules not processed for user `00a08f31638d4eaab5ca83d6f493db6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:53.858841 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ba32033-bc9d-4029-9b73-e6cb323611c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:53.960085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 678/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:27:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:58.024699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb4ad7-e755-407e-ac5d-f6bb7ccaa6b6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:58.026723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb4ad7-e755-407e-ac5d-f6bb7ccaa6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:58.026723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb4ad7-e755-407e-ac5d-f6bb7ccaa6b6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:58.028738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 674/1352] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:27:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:58.033735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc380266-9e90-48c6-a892-2894809e57c0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:58.034253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc380266-9e90-48c6-a892-2894809e57c0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:58.034372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc380266-9e90-48c6-a892-2894809e57c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:58.052753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dc380266-9e90-48c6-a892-2894809e57c0 None None] MFA Rules not processed for user `00a08f31638d4eaab5ca83d6f493db6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:58.053971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dc380266-9e90-48c6-a892-2894809e57c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:58.063183 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 679/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:27:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:27:58.502178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=13 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:27:59.607368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68393b-fc0d-496d-af4f-a73841ae64ec None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:59.608031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68393b-fc0d-496d-af4f-a73841ae64ec None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:59.608031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68393b-fc0d-496d-af4f-a73841ae64ec None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:59.609496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 675/1354] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:27:59.614589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8c3ee-98de-4faa-877b-e05804215ea2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:27:59.615035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8c3ee-98de-4faa-877b-e05804215ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:27:59.615323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8c3ee-98de-4faa-877b-e05804215ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:27:59.634281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-abe8c3ee-98de-4faa-877b-e05804215ea2 None None] MFA Rules not processed for user `00a08f31638d4eaab5ca83d6f493db6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:27:59.635552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-abe8c3ee-98de-4faa-877b-e05804215ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:27:59.644562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 680/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:27:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:03.130087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4f933-d259-451b-8ed7-af554821b6a6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:03.131881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4f933-d259-451b-8ed7-af554821b6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:03.132433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4f933-d259-451b-8ed7-af554821b6a6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:03.134348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 676/1356] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:03.140990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba724165-ca6e-4439-bb8f-9e81a3185dff None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:03.141983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba724165-ca6e-4439-bb8f-9e81a3185dff None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:03.142456 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba724165-ca6e-4439-bb8f-9e81a3185dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:03.163813 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ba724165-ca6e-4439-bb8f-9e81a3185dff None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:03.165363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ba724165-ca6e-4439-bb8f-9e81a3185dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:03.174580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 681/1357] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:03.240407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828fcce-0e2d-4332-94f7-1f2b933c6f22 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:03.241311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828fcce-0e2d-4332-94f7-1f2b933c6f22 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:03.242023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828fcce-0e2d-4332-94f7-1f2b933c6f22 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:03.245395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 677/1358] 10.222.0.27 () {54 vars in 961 bytes} [Wed Jun 29 08:28:03 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:03.253210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7250fa6-e832-49b2-9dc1-f6d3e176e2f7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:03.253781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7250fa6-e832-49b2-9dc1-f6d3e176e2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:03.254360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7250fa6-e832-49b2-9dc1-f6d3e176e2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:03.300632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:28:03.309357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f7250fa6-e832-49b2-9dc1-f6d3e176e2f7 None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:03.310583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f7250fa6-e832-49b2-9dc1-f6d3e176e2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:03.403228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 682/1359] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Jun 29 08:28:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:03.411087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828fcce-0e2d-4332-94f7-1f2b933c6f22 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:03.428318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828fcce-0e2d-4332-94f7-1f2b933c6f22 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:03.434730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a3678-7427-4055-b2c2-cea868f84daf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:03.435607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a3678-7427-4055-b2c2-cea868f84daf service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:03.436180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a3678-7427-4055-b2c2-cea868f84daf service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:03.436669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a3678-7427-4055-b2c2-cea868f84daf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:03.447130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a3678-7427-4055-b2c2-cea868f84daf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:03.449187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a3678-7427-4055-b2c2-cea868f84daf service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:03.461488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 678/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:05.668098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a019921-df19-4824-bab6-c826594f81c1 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:05.668666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a019921-df19-4824-bab6-c826594f81c1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:05.668666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a019921-df19-4824-bab6-c826594f81c1 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:05.670194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 683/1361] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:05.675385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3dfcd-aab3-4e4e-9c47-657a712bb76e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:05.675871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3dfcd-aab3-4e4e-9c47-657a712bb76e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:05.676039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3dfcd-aab3-4e4e-9c47-657a712bb76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:05.694180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-86b3dfcd-aab3-4e4e-9c47-657a712bb76e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:05.695409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-86b3dfcd-aab3-4e4e-9c47-657a712bb76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:05.703793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 679/1362] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:05.747846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a019921-df19-4824-bab6-c826594f81c1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:05.763792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a019921-df19-4824-bab6-c826594f81c1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:05.769858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0413a-f70f-4c23-85ac-f1f445d8e0a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:05.770661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0413a-f70f-4c23-85ac-f1f445d8e0a2 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:05.770969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0413a-f70f-4c23-85ac-f1f445d8e0a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:05.771242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0413a-f70f-4c23-85ac-f1f445d8e0a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:05.780653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0413a-f70f-4c23-85ac-f1f445d8e0a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:05.782247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0413a-f70f-4c23-85ac-f1f445d8e0a2 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:05.794198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 684/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:08.125869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c0318-f42d-4891-a219-a1ceb486edde None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:08.126363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c0318-f42d-4891-a219-a1ceb486edde None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:08.126363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c0318-f42d-4891-a219-a1ceb486edde None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:08.128042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 680/1364] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:08.133716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c869e-ecf4-40f6-aac0-088c2653ee03 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:08.134220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c869e-ecf4-40f6-aac0-088c2653ee03 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:08.134330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c869e-ecf4-40f6-aac0-088c2653ee03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:08.153250 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5d7c869e-ecf4-40f6-aac0-088c2653ee03 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:08.154481 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5d7c869e-ecf4-40f6-aac0-088c2653ee03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:08.164449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 685/1365] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:08.215122 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b428c-7703-4205-8199-86d7a4cdc78d None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:08.215799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b428c-7703-4205-8199-86d7a4cdc78d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:08.216297 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b428c-7703-4205-8199-86d7a4cdc78d None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:08.218225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 681/1366] 10.222.0.27 () {54 vars in 961 bytes} [Wed Jun 29 08:28:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:08.225786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde0f5e-19b4-4caf-a42c-15b9fd4a6102 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:08.226377 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde0f5e-19b4-4caf-a42c-15b9fd4a6102 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:08.226895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde0f5e-19b4-4caf-a42c-15b9fd4a6102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:08.252896 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4bde0f5e-19b4-4caf-a42c-15b9fd4a6102 None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:08.255097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4bde0f5e-19b4-4caf-a42c-15b9fd4a6102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:08.264801 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 686/1367] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Jun 29 08:28:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:08.273507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b428c-7703-4205-8199-86d7a4cdc78d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:08.288350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b428c-7703-4205-8199-86d7a4cdc78d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:08.293677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2c12c-4dde-423b-a00b-e8c3b074b016 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:08.294322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2c12c-4dde-423b-a00b-e8c3b074b016 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:08.294603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2c12c-4dde-423b-a00b-e8c3b074b016 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:08.294909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2c12c-4dde-423b-a00b-e8c3b074b016 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:08.303432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2c12c-4dde-423b-a00b-e8c3b074b016 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:08.305030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2c12c-4dde-423b-a00b-e8c3b074b016 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:08.314913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 682/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:10.442644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e86db1-4fca-410c-bef8-276fafb6abdf None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:10.443252 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e86db1-4fca-410c-bef8-276fafb6abdf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:10.443252 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e86db1-4fca-410c-bef8-276fafb6abdf None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:10.444830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 687/1369] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:10.450142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e97c-8111-46b4-a7f3-1b4038b827f2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:10.450607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e97c-8111-46b4-a7f3-1b4038b827f2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:10.450716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03e97c-8111-46b4-a7f3-1b4038b827f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:10.473886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-aa03e97c-8111-46b4-a7f3-1b4038b827f2 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:10.475139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-aa03e97c-8111-46b4-a7f3-1b4038b827f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:10.483957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 683/1370] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:10.528207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e86db1-4fca-410c-bef8-276fafb6abdf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:10.544170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e86db1-4fca-410c-bef8-276fafb6abdf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:10.550108 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719e2fea-695c-435b-a54a-5c244f70a2d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:10.551070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e2fea-695c-435b-a54a-5c244f70a2d1 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:10.551348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e2fea-695c-435b-a54a-5c244f70a2d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:10.551626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719e2fea-695c-435b-a54a-5c244f70a2d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:10.561623 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e2fea-695c-435b-a54a-5c244f70a2d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:10.563337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719e2fea-695c-435b-a54a-5c244f70a2d1 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:10.575391 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 688/1371] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:11.206471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fa66b-d015-47b2-a1f3-9b0bb140e795 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:11.206815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fa66b-d015-47b2-a1f3-9b0bb140e795 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:11.207722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6fa66b-d015-47b2-a1f3-9b0bb140e795 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:11.209700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 684/1372] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:11.216493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6068c0-6ce8-4f22-a0e5-d8d93b62cfef None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:11.216952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6068c0-6ce8-4f22-a0e5-d8d93b62cfef None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:11.217331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6068c0-6ce8-4f22-a0e5-d8d93b62cfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:11.235944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3c6068c0-6ce8-4f22-a0e5-d8d93b62cfef None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:11.237497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3c6068c0-6ce8-4f22-a0e5-d8d93b62cfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:11.245803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 689/1373] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:28:11.591486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5204b67-ba73-417a-b2e1-e2de93b9661b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:11.591486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5204b67-ba73-417a-b2e1-e2de93b9661b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:11.592143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5204b67-ba73-417a-b2e1-e2de93b9661b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:11.593065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 685/1374] 10.222.0.27 () {54 vars in 963 bytes} [Wed Jun 29 08:28:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:11.598020 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be01af-8096-48f9-94e2-bf36c8e2e030 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:11.598475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be01af-8096-48f9-94e2-bf36c8e2e030 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:11.598637 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be01af-8096-48f9-94e2-bf36c8e2e030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:11.651608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-31be01af-8096-48f9-94e2-bf36c8e2e030 None None] MFA Rules not processed for user `b60ca33edf5a47fdab6e39cc98638c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:11.652902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-31be01af-8096-48f9-94e2-bf36c8e2e030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:11.732554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 690/1375] 10.222.0.27 () {58 vars in 1060 bytes} [Wed Jun 29 08:28:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:11.733826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5204b67-ba73-417a-b2e1-e2de93b9661b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:11.747769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5204b67-ba73-417a-b2e1-e2de93b9661b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:11.752613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4c67a-d7df-4ccc-9f96-e7696ab2aea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:11.753580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4c67a-d7df-4ccc-9f96-e7696ab2aea1 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:11.754245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4c67a-d7df-4ccc-9f96-e7696ab2aea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:11.754727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4c67a-d7df-4ccc-9f96-e7696ab2aea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:11.766801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4c67a-d7df-4ccc-9f96-e7696ab2aea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:11.768805 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4c67a-d7df-4ccc-9f96-e7696ab2aea1 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:11.795039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 686/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:12.647894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8300485-0743-441b-94dc-0fb8210d2f1d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:12.648419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8300485-0743-441b-94dc-0fb8210d2f1d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:12.648542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8300485-0743-441b-94dc-0fb8210d2f1d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:12.652600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 691/1377] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:12.655575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493215d1-8d27-4ef8-8519-e8ea8745c2d5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:12.656408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493215d1-8d27-4ef8-8519-e8ea8745c2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:12.656907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493215d1-8d27-4ef8-8519-e8ea8745c2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:12.675740 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-493215d1-8d27-4ef8-8519-e8ea8745c2d5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:12.681446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-493215d1-8d27-4ef8-8519-e8ea8745c2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:12.691989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 687/1378] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:12.736094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8300485-0743-441b-94dc-0fb8210d2f1d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:12.751293 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8300485-0743-441b-94dc-0fb8210d2f1d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:12.758164 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22286bc5-50af-4a7a-8b33-0a475a1f24bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:12.758840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22286bc5-50af-4a7a-8b33-0a475a1f24bb service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:12.759124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22286bc5-50af-4a7a-8b33-0a475a1f24bb service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:12.759400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22286bc5-50af-4a7a-8b33-0a475a1f24bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:12.767946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22286bc5-50af-4a7a-8b33-0a475a1f24bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:12.769619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22286bc5-50af-4a7a-8b33-0a475a1f24bb service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:12.773521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:28:12.780718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 692/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:14.902421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93733a-a6f1-41e7-ad12-5d3cd09ae951 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:14.903013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93733a-a6f1-41e7-ad12-5d3cd09ae951 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:14.903013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf93733a-a6f1-41e7-ad12-5d3cd09ae951 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:14.904315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 688/1380] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:14.909731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f582bd2-0b99-41fe-aa16-ddfa8848a665 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:14.911147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f582bd2-0b99-41fe-aa16-ddfa8848a665 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:14.911446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f582bd2-0b99-41fe-aa16-ddfa8848a665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:14.930434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6f582bd2-0b99-41fe-aa16-ddfa8848a665 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:14.931689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6f582bd2-0b99-41fe-aa16-ddfa8848a665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:14.940094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 693/1381] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:14.986516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93733a-a6f1-41e7-ad12-5d3cd09ae951 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:15.004521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf93733a-a6f1-41e7-ad12-5d3cd09ae951 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:15.010910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4de117-f95c-4dd4-9502-a25679afcf91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:15.011565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4de117-f95c-4dd4-9502-a25679afcf91 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:15.011841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4de117-f95c-4dd4-9502-a25679afcf91 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:15.012103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4de117-f95c-4dd4-9502-a25679afcf91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:15.021709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4de117-f95c-4dd4-9502-a25679afcf91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:15.023352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4de117-f95c-4dd4-9502-a25679afcf91 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:15.033311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 689/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:17.106483 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f98426-b7bd-4ebc-836f-c582ab814a85 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:17.107087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f98426-b7bd-4ebc-836f-c582ab814a85 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:17.107312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f98426-b7bd-4ebc-836f-c582ab814a85 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:17.108985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 694/1383] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:17.114816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf4d3-0d40-4a67-9023-ee2d701670a9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:17.115299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf4d3-0d40-4a67-9023-ee2d701670a9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:17.115551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bf4d3-0d40-4a67-9023-ee2d701670a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:17.135565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cc9bf4d3-0d40-4a67-9023-ee2d701670a9 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:17.136767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cc9bf4d3-0d40-4a67-9023-ee2d701670a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:17.145722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 690/1384] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:17.188014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f98426-b7bd-4ebc-836f-c582ab814a85 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:17.202493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f98426-b7bd-4ebc-836f-c582ab814a85 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:17.208596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9942dd4-daca-4839-9648-9bb17fce552c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:17.209497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9942dd4-daca-4839-9648-9bb17fce552c service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:17.209803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9942dd4-daca-4839-9648-9bb17fce552c service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:17.210043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9942dd4-daca-4839-9648-9bb17fce552c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:17.220370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9942dd4-daca-4839-9648-9bb17fce552c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:17.221884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9942dd4-daca-4839-9648-9bb17fce552c service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:17.232425 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 695/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:19.259437 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6fda5-a3c9-4b1b-86fe-3fc6d6fca62b None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:19.259968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6fda5-a3c9-4b1b-86fe-3fc6d6fca62b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:19.259968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6fda5-a3c9-4b1b-86fe-3fc6d6fca62b None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:19.261366 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 691/1386] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:19.266611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d03cd-0eef-42e9-ac33-f8ab6bf2d36b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:19.267052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d03cd-0eef-42e9-ac33-f8ab6bf2d36b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:19.267269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d03cd-0eef-42e9-ac33-f8ab6bf2d36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:19.286553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-133d03cd-0eef-42e9-ac33-f8ab6bf2d36b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:19.287744 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-133d03cd-0eef-42e9-ac33-f8ab6bf2d36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:19.296871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 696/1387] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:28:19.341467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6fda5-a3c9-4b1b-86fe-3fc6d6fca62b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:19.357317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6fda5-a3c9-4b1b-86fe-3fc6d6fca62b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:19.362823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f444b-a566-43e0-aed7-8a7da2276ce0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:19.364064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f444b-a566-43e0-aed7-8a7da2276ce0 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:19.364350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f444b-a566-43e0-aed7-8a7da2276ce0 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:19.364628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f444b-a566-43e0-aed7-8a7da2276ce0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:19.372955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f444b-a566-43e0-aed7-8a7da2276ce0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:19.374550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f444b-a566-43e0-aed7-8a7da2276ce0 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:19.385043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 692/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:21.551612 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b84b-0cb2-4d7d-b980-b594f65dce1f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:21.552527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b84b-0cb2-4d7d-b980-b594f65dce1f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:21.552527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b84b-0cb2-4d7d-b980-b594f65dce1f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:21.553617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 697/1389] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:21.560733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d27d6-847a-4123-8e01-5d0d490c1adb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:21.561257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d27d6-847a-4123-8e01-5d0d490c1adb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:21.561732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d27d6-847a-4123-8e01-5d0d490c1adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:21.593188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d48d27d6-847a-4123-8e01-5d0d490c1adb None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:21.595370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d48d27d6-847a-4123-8e01-5d0d490c1adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:21.610145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 693/1390] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:21.656847 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9b84b-0cb2-4d7d-b980-b594f65dce1f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:21.673931 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9b84b-0cb2-4d7d-b980-b594f65dce1f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:21.680459 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee1ea32-c8e3-44bd-ac4f-45d60cd6acc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:21.681083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1ea32-c8e3-44bd-ac4f-45d60cd6acc7 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:21.681362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1ea32-c8e3-44bd-ac4f-45d60cd6acc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:21.681631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee1ea32-c8e3-44bd-ac4f-45d60cd6acc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:21.690835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1ea32-c8e3-44bd-ac4f-45d60cd6acc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:21.692823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee1ea32-c8e3-44bd-ac4f-45d60cd6acc7 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:21.703553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 698/1391] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:23.802065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca87d0-4588-4826-90ba-ccf029ea6869 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:23.803035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca87d0-4588-4826-90ba-ccf029ea6869 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:23.803035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca87d0-4588-4826-90ba-ccf029ea6869 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:23.804066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 694/1392] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:28:23.810516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3d5d-4584-47dc-b20a-ad2da2be8463 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:23.811072 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3d5d-4584-47dc-b20a-ad2da2be8463 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:23.811205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef3d5d-4584-47dc-b20a-ad2da2be8463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:23.833443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-08ef3d5d-4584-47dc-b20a-ad2da2be8463 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:23.834745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-08ef3d5d-4584-47dc-b20a-ad2da2be8463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:23.844078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 699/1393] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:28:23.896718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca87d0-4588-4826-90ba-ccf029ea6869 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:23.922901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca87d0-4588-4826-90ba-ccf029ea6869 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:23.932848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d281b0e-2688-41a3-86c1-b7e15e5494ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:23.933961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d281b0e-2688-41a3-86c1-b7e15e5494ed service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:23.934497 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d281b0e-2688-41a3-86c1-b7e15e5494ed service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:23.934986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d281b0e-2688-41a3-86c1-b7e15e5494ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:23.949137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d281b0e-2688-41a3-86c1-b7e15e5494ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:23.950900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d281b0e-2688-41a3-86c1-b7e15e5494ed service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:23.961916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 695/1394] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:26.355023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326acc8-3d72-48c4-8cf6-52fe714d432a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:26.355604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326acc8-3d72-48c4-8cf6-52fe714d432a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:26.355693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326acc8-3d72-48c4-8cf6-52fe714d432a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:26.357042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 700/1395] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:26.362374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba388a-9ab4-4966-8b6f-b2a5116c11ee None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:26.362819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba388a-9ab4-4966-8b6f-b2a5116c11ee None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:26.363053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba388a-9ab4-4966-8b6f-b2a5116c11ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:26.381685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-baba388a-9ab4-4966-8b6f-b2a5116c11ee None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:26.382907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-baba388a-9ab4-4966-8b6f-b2a5116c11ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:26.391893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 696/1396] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:26.435643 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326acc8-3d72-48c4-8cf6-52fe714d432a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:26.451232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326acc8-3d72-48c4-8cf6-52fe714d432a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:26.459344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a44d8b-0abb-4d02-a2ad-6df4ea862454 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:26.460005 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a44d8b-0abb-4d02-a2ad-6df4ea862454 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:26.460254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a44d8b-0abb-4d02-a2ad-6df4ea862454 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:26.460524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a44d8b-0abb-4d02-a2ad-6df4ea862454 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:26.470024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a44d8b-0abb-4d02-a2ad-6df4ea862454 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:26.471687 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a44d8b-0abb-4d02-a2ad-6df4ea862454 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:26.481905 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 701/1397] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:27.028151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc20705-81b9-4267-8f89-4868ee6fae49 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:27.028432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc20705-81b9-4267-8f89-4868ee6fae49 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:27.028698 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc20705-81b9-4267-8f89-4868ee6fae49 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:27.030099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 697/1398] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:27.035501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf211357-4579-457c-8bd7-318858eed622 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:27.035791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf211357-4579-457c-8bd7-318858eed622 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:27.036069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf211357-4579-457c-8bd7-318858eed622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:27.054229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bf211357-4579-457c-8bd7-318858eed622 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:27.055484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bf211357-4579-457c-8bd7-318858eed622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:27.064164 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 702/1399] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:27.119387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a4219-12d0-478d-8f31-15061eb451de None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:27.119851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a4219-12d0-478d-8f31-15061eb451de None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:27.120191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a4219-12d0-478d-8f31-15061eb451de None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:27.121616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 698/1400] 10.222.0.27 () {54 vars in 962 bytes} [Wed Jun 29 08:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:27.126949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf804dcb-2875-4b08-a011-6d81574ade19 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:27.127315 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf804dcb-2875-4b08-a011-6d81574ade19 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:27.127677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf804dcb-2875-4b08-a011-6d81574ade19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:27.177604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bf804dcb-2875-4b08-a011-6d81574ade19 None None] MFA Rules not processed for user `fd09453b5db645d7b03522f91effb8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:27.178834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bf804dcb-2875-4b08-a011-6d81574ade19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:27.241184 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 703/1401] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Jun 29 08:28:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:27.248844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a4219-12d0-478d-8f31-15061eb451de None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:27.260203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a4219-12d0-478d-8f31-15061eb451de None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:27.263986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc3f6a-8c9f-4ff1-8555-fdb66c96a290 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd09453b5db645d7b03522f91effb8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:27.264680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc3f6a-8c9f-4ff1-8555-fdb66c96a290 service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:27.264958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc3f6a-8c9f-4ff1-8555-fdb66c96a290 service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:27.265210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc3f6a-8c9f-4ff1-8555-fdb66c96a290 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:27.274682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc3f6a-8c9f-4ff1-8555-fdb66c96a290 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:27.276132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc3f6a-8c9f-4ff1-8555-fdb66c96a290 service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:27.285602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 699/1402] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:28.008143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd483a-a054-4519-bc88-c5f279df49e5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:28.008614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd483a-a054-4519-bc88-c5f279df49e5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:28.008703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd483a-a054-4519-bc88-c5f279df49e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:28.059553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-02bd483a-a054-4519-bc88-c5f279df49e5 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:28.060758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-02bd483a-a054-4519-bc88-c5f279df49e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:28.102030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:28:28.126474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 704/1403] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:28:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:28.168266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606d27d-88ee-4681-a4b2-ab5143c893f5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:28.168743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606d27d-88ee-4681-a4b2-ab5143c893f5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:28.168947 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606d27d-88ee-4681-a4b2-ab5143c893f5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:28.170410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 700/1404] 10.222.0.27 () {54 vars in 942 bytes} [Wed Jun 29 08:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:28.177163 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f3901-a6f0-40ad-a57a-c0dcf99430b5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:28.177474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f3901-a6f0-40ad-a57a-c0dcf99430b5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:28.177769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f3901-a6f0-40ad-a57a-c0dcf99430b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:28.230142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bf7f3901-a6f0-40ad-a57a-c0dcf99430b5 None None] MFA Rules not processed for user `3b38411347754c058eb6640829078e1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:28.231580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bf7f3901-a6f0-40ad-a57a-c0dcf99430b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:28.305395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 705/1405] 10.222.0.27 () {58 vars in 1039 bytes} [Wed Jun 29 08:28:28 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:28.314869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606d27d-88ee-4681-a4b2-ab5143c893f5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:28.328338 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606d27d-88ee-4681-a4b2-ab5143c893f5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:28.333620 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35cf716-5878-4639-9f61-76eace5afb40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:28.334310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cf716-5878-4639-9f61-76eace5afb40 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:28.334593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cf716-5878-4639-9f61-76eace5afb40 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:28.334926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35cf716-5878-4639-9f61-76eace5afb40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:28.343697 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35cf716-5878-4639-9f61-76eace5afb40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:28.345278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35cf716-5878-4639-9f61-76eace5afb40 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:28.353572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 701/1406] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:28:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:28:28.549184 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31711bbe-aec6-4523-bfa3-df30fe11bfe4 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:28.549714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31711bbe-aec6-4523-bfa3-df30fe11bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:28.550009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31711bbe-aec6-4523-bfa3-df30fe11bfe4 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:28.551545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 706/1407] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:28.556872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc14567-48e5-4c8b-824a-581a55031f47 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:28.557333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc14567-48e5-4c8b-824a-581a55031f47 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:28.557783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc14567-48e5-4c8b-824a-581a55031f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:28.565274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=67 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:28:28.579858 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4fc14567-48e5-4c8b-824a-581a55031f47 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:28.581280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4fc14567-48e5-4c8b-824a-581a55031f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:28.602721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 702/1408] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:28.639918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31711bbe-aec6-4523-bfa3-df30fe11bfe4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:28.655282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31711bbe-aec6-4523-bfa3-df30fe11bfe4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:28.660807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d45516-557c-4d96-b8b9-8e2ea2151360 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:28.661495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d45516-557c-4d96-b8b9-8e2ea2151360 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:28.661790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d45516-557c-4d96-b8b9-8e2ea2151360 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:28.662052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d45516-557c-4d96-b8b9-8e2ea2151360 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:28.672638 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d45516-557c-4d96-b8b9-8e2ea2151360 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:28.674263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d45516-557c-4d96-b8b9-8e2ea2151360 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:28.684626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 707/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:38.574434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:28:38.680173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=59 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:28:50.308721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d26276-32fc-4955-9b0d-47a8e8fb21f8 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:50.309385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d26276-32fc-4955-9b0d-47a8e8fb21f8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:50.309385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d26276-32fc-4955-9b0d-47a8e8fb21f8 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:50.310643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 703/1410] 10.222.0.27 () {54 vars in 918 bytes} [Wed Jun 29 08:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:50.316558 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514334a-4918-4c47-a462-64d606ea8eb0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:50.316985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514334a-4918-4c47-a462-64d606ea8eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:50.317183 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514334a-4918-4c47-a462-64d606ea8eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:50.338054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4514334a-4918-4c47-a462-64d606ea8eb0 None None] MFA Rules not processed for user `00a08f31638d4eaab5ca83d6f493db6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:50.339655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4514334a-4918-4c47-a462-64d606ea8eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:50.350336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 708/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Jun 29 08:28:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:50.366130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf882e03-ef92-493e-b534-274465773da9 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:50.366553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf882e03-ef92-493e-b534-274465773da9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:50.367046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf882e03-ef92-493e-b534-274465773da9 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:50.369057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 704/1412] 10.222.0.27 () {54 vars in 918 bytes} [Wed Jun 29 08:28:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:50.373770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b6479-8e3e-4665-b5ae-0075ad9e9d37 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:50.374068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b6479-8e3e-4665-b5ae-0075ad9e9d37 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:50.374351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b6479-8e3e-4665-b5ae-0075ad9e9d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:50.393477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d20b6479-8e3e-4665-b5ae-0075ad9e9d37 None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:50.395354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d20b6479-8e3e-4665-b5ae-0075ad9e9d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:50.405462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 709/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Wed Jun 29 08:28:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:53.049731 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12b6b-5386-468b-ae59-d450453a31c6 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:53.050608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12b6b-5386-468b-ae59-d450453a31c6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:53.050608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b12b6b-5386-468b-ae59-d450453a31c6 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:53.051881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:28:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:53.058371 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8bf0e-9fb6-4154-acd1-c5ed4db84f2b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:53.059130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8bf0e-9fb6-4154-acd1-c5ed4db84f2b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:53.059282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8bf0e-9fb6-4154-acd1-c5ed4db84f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:53.084526 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-48c8bf0e-9fb6-4154-acd1-c5ed4db84f2b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:53.085759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-48c8bf0e-9fb6-4154-acd1-c5ed4db84f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:53.095047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:53.143202 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b12b6b-5386-468b-ae59-d450453a31c6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:53.160602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b12b6b-5386-468b-ae59-d450453a31c6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:53.167091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075c990-b66c-412b-b51f-129b8037e05e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:53.167729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075c990-b66c-412b-b51f-129b8037e05e service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:53.168020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075c990-b66c-412b-b51f-129b8037e05e service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:53.168295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075c990-b66c-412b-b51f-129b8037e05e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:53.176731 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075c990-b66c-412b-b51f-129b8037e05e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:53.178356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c075c990-b66c-412b-b51f-129b8037e05e service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:53.188374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 706/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:54.115584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2128d0e-5db5-45e3-b624-b3b34f8f2b7e None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:54.116487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2128d0e-5db5-45e3-b624-b3b34f8f2b7e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:54.116487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2128d0e-5db5-45e3-b624-b3b34f8f2b7e None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:54.118741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:28:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:54.124841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9082fce-b1da-4298-b06c-26a6293b40fe None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:54.125311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9082fce-b1da-4298-b06c-26a6293b40fe None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:54.125415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9082fce-b1da-4298-b06c-26a6293b40fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:54.146409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d9082fce-b1da-4298-b06c-26a6293b40fe None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:54.147660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d9082fce-b1da-4298-b06c-26a6293b40fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:54.157020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:54.311627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2128d0e-5db5-45e3-b624-b3b34f8f2b7e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:54.331091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2128d0e-5db5-45e3-b624-b3b34f8f2b7e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:54.341811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c551a-3527-4370-a061-2008808f90cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:54.342892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c551a-3527-4370-a061-2008808f90cc service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:54.343362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c551a-3527-4370-a061-2008808f90cc service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:54.343821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c551a-3527-4370-a061-2008808f90cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:54.358982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c551a-3527-4370-a061-2008808f90cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:54.361613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c551a-3527-4370-a061-2008808f90cc service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:54.378926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 712/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:55.272510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd69aa-c106-4f7e-b0fb-36502cd991cb None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:55.273102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd69aa-c106-4f7e-b0fb-36502cd991cb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:55.273102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd69aa-c106-4f7e-b0fb-36502cd991cb None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:55.274529 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:28:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:55.281691 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b707-9074-41c8-bb00-dcd8e8d35937 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:55.282126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b707-9074-41c8-bb00-dcd8e8d35937 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:55.282220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494b707-9074-41c8-bb00-dcd8e8d35937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:55.303225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7494b707-9074-41c8-bb00-dcd8e8d35937 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:55.304477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7494b707-9074-41c8-bb00-dcd8e8d35937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:55.313809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:55.463353 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd69aa-c106-4f7e-b0fb-36502cd991cb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:55.479309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd69aa-c106-4f7e-b0fb-36502cd991cb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:55.485180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ca20c-452a-4e62-a4fd-531f0b8fd2e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:55.485821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ca20c-452a-4e62-a4fd-531f0b8fd2e2 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:55.486097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ca20c-452a-4e62-a4fd-531f0b8fd2e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:55.486374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ca20c-452a-4e62-a4fd-531f0b8fd2e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:55.495276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ca20c-452a-4e62-a4fd-531f0b8fd2e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:55.496860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ca20c-452a-4e62-a4fd-531f0b8fd2e2 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:55.506100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 709/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:28:56.393542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7538ce-4258-40a2-95d0-2c557a551781 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:56.394151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7538ce-4258-40a2-95d0-2c557a551781 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:56.394151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7538ce-4258-40a2-95d0-2c557a551781 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:56.395530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:56.401838 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0407154-3dde-41e4-8d45-fa55902dcaa1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:56.402577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0407154-3dde-41e4-8d45-fa55902dcaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:56.402792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0407154-3dde-41e4-8d45-fa55902dcaa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:56.425644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e0407154-3dde-41e4-8d45-fa55902dcaa1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:56.427263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e0407154-3dde-41e4-8d45-fa55902dcaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:56.436996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:56.597128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7538ce-4258-40a2-95d0-2c557a551781 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:56.621457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7538ce-4258-40a2-95d0-2c557a551781 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:56.631036 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681e65c-5d24-4536-ab26-52c5a348a2ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:56.631677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681e65c-5d24-4536-ab26-52c5a348a2ca service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:56.631958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681e65c-5d24-4536-ab26-52c5a348a2ca service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:56.632230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681e65c-5d24-4536-ab26-52c5a348a2ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:56.645518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681e65c-5d24-4536-ab26-52c5a348a2ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:56.647885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681e65c-5d24-4536-ab26-52c5a348a2ca service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:56.662898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 715/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:57.518801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e493b8-ca5a-4db5-8981-b638b4183953 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:57.519689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e493b8-ca5a-4db5-8981-b638b4183953 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:57.519689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e493b8-ca5a-4db5-8981-b638b4183953 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:57.522012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:28:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:57.528489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd16a55-9a7d-4729-b98a-0c3e39819813 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:57.528930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd16a55-9a7d-4729-b98a-0c3e39819813 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:57.529062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd16a55-9a7d-4729-b98a-0c3e39819813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:57.556945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-dcd16a55-9a7d-4729-b98a-0c3e39819813 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:57.558159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-dcd16a55-9a7d-4729-b98a-0c3e39819813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:57.568159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:57.722047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e493b8-ca5a-4db5-8981-b638b4183953 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:57.744647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e493b8-ca5a-4db5-8981-b638b4183953 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:57.751210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9970e-c847-4638-87cd-3b9fc6241bc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:57.751854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9970e-c847-4638-87cd-3b9fc6241bc0 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:57.752137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9970e-c847-4638-87cd-3b9fc6241bc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:57.752414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9970e-c847-4638-87cd-3b9fc6241bc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:57.765227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9970e-c847-4638-87cd-3b9fc6241bc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:57.766892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9970e-c847-4638-87cd-3b9fc6241bc0 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:57.776534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 712/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:58.595268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e5b07-f54f-46c3-b979-f8dc17c2066c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:58.596170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e5b07-f54f-46c3-b979-f8dc17c2066c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:58.596170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e5b07-f54f-46c3-b979-f8dc17c2066c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:58.598290 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:28:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:58.604070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965c755-40e2-4b2e-8d8a-aa0ac5b1d6e2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:58.604541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965c755-40e2-4b2e-8d8a-aa0ac5b1d6e2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:58.604645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965c755-40e2-4b2e-8d8a-aa0ac5b1d6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:58.623870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3965c755-40e2-4b2e-8d8a-aa0ac5b1d6e2 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:58.625073 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3965c755-40e2-4b2e-8d8a-aa0ac5b1d6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:58.634585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:58.786073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e5b07-f54f-46c3-b979-f8dc17c2066c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:58.802726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e5b07-f54f-46c3-b979-f8dc17c2066c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:58.809485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbd02c-8814-4e89-bb79-eed2d4c563a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:58.810128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd02c-8814-4e89-bb79-eed2d4c563a4 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:58.810408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd02c-8814-4e89-bb79-eed2d4c563a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:58.810659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd02c-8814-4e89-bb79-eed2d4c563a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:58.822317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbd02c-8814-4e89-bb79-eed2d4c563a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:58.823947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbd02c-8814-4e89-bb79-eed2d4c563a4 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:58.833950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 718/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:28:59.644165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e283-3fb1-4282-8753-2103a6495405 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:59.644765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e283-3fb1-4282-8753-2103a6495405 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:59.644765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e283-3fb1-4282-8753-2103a6495405 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:59.646094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:28:59.652636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e1a20-379c-4905-a369-759deaa30cc9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:59.653333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e1a20-379c-4905-a369-759deaa30cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:59.653468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e1a20-379c-4905-a369-759deaa30cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:59.675551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d83e1a20-379c-4905-a369-759deaa30cc9 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:28:59.676771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d83e1a20-379c-4905-a369-759deaa30cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:28:59.686464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:28:59.845588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e283-3fb1-4282-8753-2103a6495405 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:28:59.870825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e283-3fb1-4282-8753-2103a6495405 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:28:59.877522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b21dc4-d552-45b5-a7b8-553cc6a13feb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:28:59.878159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b21dc4-d552-45b5-a7b8-553cc6a13feb service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:28:59.878436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b21dc4-d552-45b5-a7b8-553cc6a13feb service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:28:59.878729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b21dc4-d552-45b5-a7b8-553cc6a13feb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:28:59.888484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b21dc4-d552-45b5-a7b8-553cc6a13feb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:28:59.890084 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b21dc4-d552-45b5-a7b8-553cc6a13feb service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:28:59.900260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 715/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:00.703938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979da93-de70-40bb-9556-9ba9ef2f72a2 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:00.704888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979da93-de70-40bb-9556-9ba9ef2f72a2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:00.704888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979da93-de70-40bb-9556-9ba9ef2f72a2 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:00.707077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:29:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:00.713351 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caeafc-a526-4124-827e-bc78bbccb73d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:00.713827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caeafc-a526-4124-827e-bc78bbccb73d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:00.713827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14caeafc-a526-4124-827e-bc78bbccb73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:00.741556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-14caeafc-a526-4124-827e-bc78bbccb73d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:00.742783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-14caeafc-a526-4124-827e-bc78bbccb73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:00.752668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:00.911131 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979da93-de70-40bb-9556-9ba9ef2f72a2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:00.927724 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979da93-de70-40bb-9556-9ba9ef2f72a2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:00.934167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c3182-42d0-4ae0-b6e5-e663daa9a6ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:00.934837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c3182-42d0-4ae0-b6e5-e663daa9a6ba service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:00.935119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c3182-42d0-4ae0-b6e5-e663daa9a6ba service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:00.935406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c3182-42d0-4ae0-b6e5-e663daa9a6ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:00.944674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c3182-42d0-4ae0-b6e5-e663daa9a6ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:00.946263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c3182-42d0-4ae0-b6e5-e663daa9a6ba service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:00.957446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 721/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:29:01.782396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f769e9-5523-4d59-9748-3a9e169d92a3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:01.783204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f769e9-5523-4d59-9748-3a9e169d92a3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:01.783204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f769e9-5523-4d59-9748-3a9e169d92a3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:01.784689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:29:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:01.791044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f5087-9b8d-41d1-b501-54155c02c285 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:01.791518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f5087-9b8d-41d1-b501-54155c02c285 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:01.791632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f5087-9b8d-41d1-b501-54155c02c285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:01.811987 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-439f5087-9b8d-41d1-b501-54155c02c285 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:01.813194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-439f5087-9b8d-41d1-b501-54155c02c285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:01.821592 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:01.970973 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f769e9-5523-4d59-9748-3a9e169d92a3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:01.994600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f769e9-5523-4d59-9748-3a9e169d92a3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:02.001319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71f555-fa20-4886-a226-567c02daeade service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:02.001967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f555-fa20-4886-a226-567c02daeade service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:02.002253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f555-fa20-4886-a226-567c02daeade service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:02.002523 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f555-fa20-4886-a226-567c02daeade service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:02.011523 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71f555-fa20-4886-a226-567c02daeade service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:02.013120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71f555-fa20-4886-a226-567c02daeade service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:02.022952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 718/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:03.970441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497f537-8ce9-413b-96c9-2165c8911d0f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:03.971442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497f537-8ce9-413b-96c9-2165c8911d0f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:03.971442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497f537-8ce9-413b-96c9-2165c8911d0f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:03.973557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:29:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:03.980509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad438f-97de-4dad-8ea7-1dd35ed5cd9f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:03.980955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad438f-97de-4dad-8ea7-1dd35ed5cd9f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:03.981066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad438f-97de-4dad-8ea7-1dd35ed5cd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:04.003322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0aad438f-97de-4dad-8ea7-1dd35ed5cd9f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:04.004510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0aad438f-97de-4dad-8ea7-1dd35ed5cd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:04.015460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:04.059975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497f537-8ce9-413b-96c9-2165c8911d0f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:04.072683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497f537-8ce9-413b-96c9-2165c8911d0f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:04.077085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c51c5-a1e3-4ae5-8cf0-d0ff0c807373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:04.077727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c51c5-a1e3-4ae5-8cf0-d0ff0c807373 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:04.078003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c51c5-a1e3-4ae5-8cf0-d0ff0c807373 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:04.078285 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c51c5-a1e3-4ae5-8cf0-d0ff0c807373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:04.088252 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c51c5-a1e3-4ae5-8cf0-d0ff0c807373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:04.090011 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c51c5-a1e3-4ae5-8cf0-d0ff0c807373 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:04.099865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 724/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:06.282984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20a96d-aa6c-4a69-8b88-106d6e933ac5 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:06.283597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20a96d-aa6c-4a69-8b88-106d6e933ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:06.283597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20a96d-aa6c-4a69-8b88-106d6e933ac5 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:06.284992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:29:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:06.290463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3925e-0ee1-4575-aee2-6de989ecdd66 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:06.290945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3925e-0ee1-4575-aee2-6de989ecdd66 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:06.291053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3925e-0ee1-4575-aee2-6de989ecdd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:06.309011 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bcf3925e-0ee1-4575-aee2-6de989ecdd66 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:06.310191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bcf3925e-0ee1-4575-aee2-6de989ecdd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:06.318394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:06.360974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20a96d-aa6c-4a69-8b88-106d6e933ac5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:06.377718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20a96d-aa6c-4a69-8b88-106d6e933ac5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:06.382127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266700b7-1590-4fdb-b4e7-dd25777bef60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:06.382797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266700b7-1590-4fdb-b4e7-dd25777bef60 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:06.383066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266700b7-1590-4fdb-b4e7-dd25777bef60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:06.383364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266700b7-1590-4fdb-b4e7-dd25777bef60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:06.392977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266700b7-1590-4fdb-b4e7-dd25777bef60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:06.394401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266700b7-1590-4fdb-b4e7-dd25777bef60 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:06.404031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 721/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:08.821833 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeb1c8-15d9-4f60-b599-58112f114964 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:08.822360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeb1c8-15d9-4f60-b599-58112f114964 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:08.822360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeb1c8-15d9-4f60-b599-58112f114964 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:08.823807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:08.829974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799a508-d15a-4cfb-96fd-16c0619e5333 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:08.830449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799a508-d15a-4cfb-96fd-16c0619e5333 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:08.830539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799a508-d15a-4cfb-96fd-16c0619e5333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:08.851328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5799a508-d15a-4cfb-96fd-16c0619e5333 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:08.852561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5799a508-d15a-4cfb-96fd-16c0619e5333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:08.862158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:08.903710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effeb1c8-15d9-4f60-b599-58112f114964 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:08.923602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effeb1c8-15d9-4f60-b599-58112f114964 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:08.929739 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cc29e-b2f9-4e56-9f83-181cb34a5d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:08.930383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cc29e-b2f9-4e56-9f83-181cb34a5d66 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:08.930665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cc29e-b2f9-4e56-9f83-181cb34a5d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:08.930962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cc29e-b2f9-4e56-9f83-181cb34a5d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:08.940529 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cc29e-b2f9-4e56-9f83-181cb34a5d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:08.942001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cc29e-b2f9-4e56-9f83-181cb34a5d66 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:08.952038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 727/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:11.150256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54642b61-bf0d-42be-bffe-7e9526a7b9b8 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:11.150812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54642b61-bf0d-42be-bffe-7e9526a7b9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:11.150812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54642b61-bf0d-42be-bffe-7e9526a7b9b8 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:11.152209 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:11.157671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124565d3-b8a7-4848-80c9-d6ea303f22bd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:11.158116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124565d3-b8a7-4848-80c9-d6ea303f22bd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:11.158212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124565d3-b8a7-4848-80c9-d6ea303f22bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:11.177637 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-124565d3-b8a7-4848-80c9-d6ea303f22bd None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:11.178871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-124565d3-b8a7-4848-80c9-d6ea303f22bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:11.187593 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:11.228153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54642b61-bf0d-42be-bffe-7e9526a7b9b8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:11.239725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54642b61-bf0d-42be-bffe-7e9526a7b9b8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:11.243487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a6efd-e7f0-478b-94f4-f179a73b05c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:11.244159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a6efd-e7f0-478b-94f4-f179a73b05c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:11.244437 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a6efd-e7f0-478b-94f4-f179a73b05c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:11.244706 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a6efd-e7f0-478b-94f4-f179a73b05c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:11.254467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a6efd-e7f0-478b-94f4-f179a73b05c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:11.255974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a6efd-e7f0-478b-94f4-f179a73b05c7 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:11.267064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 724/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:13.358349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc79eb1-561c-4dd2-999a-433c98a38e0b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:13.358966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc79eb1-561c-4dd2-999a-433c98a38e0b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:13.358966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc79eb1-561c-4dd2-999a-433c98a38e0b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:13.360334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:13.366945 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08fd80-ba45-4b0a-8319-554cbd75def1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:13.367421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08fd80-ba45-4b0a-8319-554cbd75def1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:13.367791 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08fd80-ba45-4b0a-8319-554cbd75def1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:13.395532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ec08fd80-ba45-4b0a-8319-554cbd75def1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:13.397428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ec08fd80-ba45-4b0a-8319-554cbd75def1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:13.406730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:13.448248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc79eb1-561c-4dd2-999a-433c98a38e0b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:13.466244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc79eb1-561c-4dd2-999a-433c98a38e0b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:13.470608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd20ce-16cc-4a8a-af14-239b12b751d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:13.471310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd20ce-16cc-4a8a-af14-239b12b751d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:13.471595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd20ce-16cc-4a8a-af14-239b12b751d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:13.471872 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd20ce-16cc-4a8a-af14-239b12b751d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:13.481713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd20ce-16cc-4a8a-af14-239b12b751d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:13.483200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfd20ce-16cc-4a8a-af14-239b12b751d9 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:13.492887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 730/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:15.569368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2eb3a-e763-4485-8485-6cbd5c14f3fb None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:15.570127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2eb3a-e763-4485-8485-6cbd5c14f3fb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:15.570127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2eb3a-e763-4485-8485-6cbd5c14f3fb None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:15.571396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:15.576996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948f74-4947-4c3e-9d66-c2685fab7e47 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:15.577421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948f74-4947-4c3e-9d66-c2685fab7e47 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:15.577534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37948f74-4947-4c3e-9d66-c2685fab7e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:15.597677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-37948f74-4947-4c3e-9d66-c2685fab7e47 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:15.598904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-37948f74-4947-4c3e-9d66-c2685fab7e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:15.607624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:15.644997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2eb3a-e763-4485-8485-6cbd5c14f3fb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:15.657372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2eb3a-e763-4485-8485-6cbd5c14f3fb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:15.661642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129c7d8-120c-483c-af22-fcd86307895c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:15.662307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129c7d8-120c-483c-af22-fcd86307895c service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:15.662585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129c7d8-120c-483c-af22-fcd86307895c service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:15.662883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129c7d8-120c-483c-af22-fcd86307895c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:15.672394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129c7d8-120c-483c-af22-fcd86307895c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:15.673817 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129c7d8-120c-483c-af22-fcd86307895c service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:15.683752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 727/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:17.772902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a404-9255-4128-b36d-8994eab6acdf None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:17.773454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a404-9255-4128-b36d-8994eab6acdf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:17.773573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045a404-9255-4128-b36d-8994eab6acdf None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:17.774941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:17.781135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b0f11-f932-4e74-a31a-be4091eba52d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:17.781780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b0f11-f932-4e74-a31a-be4091eba52d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:17.781943 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b0f11-f932-4e74-a31a-be4091eba52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:17.807313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-874b0f11-f932-4e74-a31a-be4091eba52d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:17.808505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-874b0f11-f932-4e74-a31a-be4091eba52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:17.817078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:17.854753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045a404-9255-4128-b36d-8994eab6acdf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:17.866652 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045a404-9255-4128-b36d-8994eab6acdf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:17.870363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2d861-3802-46f4-b8f7-0efdf2078701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:17.871050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2d861-3802-46f4-b8f7-0efdf2078701 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:17.871335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2d861-3802-46f4-b8f7-0efdf2078701 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:17.871611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2d861-3802-46f4-b8f7-0efdf2078701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:17.881779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2d861-3802-46f4-b8f7-0efdf2078701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:17.883327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2d861-3802-46f4-b8f7-0efdf2078701 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:17.892936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 733/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:19.966384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b159e10-8d34-43bc-80e2-810a8831e16c None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:19.967334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b159e10-8d34-43bc-80e2-810a8831e16c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:19.967334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b159e10-8d34-43bc-80e2-810a8831e16c None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:19.969429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:29:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:19.976201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f5820-60b4-4a21-af85-48c8306a3992 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:19.976679 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f5820-60b4-4a21-af85-48c8306a3992 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:19.976777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f5820-60b4-4a21-af85-48c8306a3992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:19.987471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=79 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:29:20.002669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-292f5820-60b4-4a21-af85-48c8306a3992 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:20.003919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-292f5820-60b4-4a21-af85-48c8306a3992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:20.013590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:29:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:20.055646 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b159e10-8d34-43bc-80e2-810a8831e16c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:20.065032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=73 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:29:20.075535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b159e10-8d34-43bc-80e2-810a8831e16c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:20.081777 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101ef37-e890-4e66-a271-71e4a47ff62d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:20.082720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101ef37-e890-4e66-a271-71e4a47ff62d service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:20.083109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101ef37-e890-4e66-a271-71e4a47ff62d service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:20.083449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101ef37-e890-4e66-a271-71e4a47ff62d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:20.094152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101ef37-e890-4e66-a271-71e4a47ff62d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:20.095632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101ef37-e890-4e66-a271-71e4a47ff62d service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:20.105275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 730/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:22.149029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa5a02-58bf-440e-a230-91662454049b None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:22.149938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa5a02-58bf-440e-a230-91662454049b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:22.149938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa5a02-58bf-440e-a230-91662454049b None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:22.152202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:29:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:22.157799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd4e5a-2f40-427d-9642-7ee93f4e16f5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:22.158265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd4e5a-2f40-427d-9642-7ee93f4e16f5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:22.158374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd4e5a-2f40-427d-9642-7ee93f4e16f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:22.176857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f7fd4e5a-2f40-427d-9642-7ee93f4e16f5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:22.178049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f7fd4e5a-2f40-427d-9642-7ee93f4e16f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:22.187437 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:22.227681 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa5a02-58bf-440e-a230-91662454049b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:22.244700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa5a02-58bf-440e-a230-91662454049b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:22.248894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee8a67-005d-4d94-b27f-3b1b776068de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:22.249534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee8a67-005d-4d94-b27f-3b1b776068de service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:22.249815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee8a67-005d-4d94-b27f-3b1b776068de service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:22.250083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee8a67-005d-4d94-b27f-3b1b776068de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:22.263730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee8a67-005d-4d94-b27f-3b1b776068de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:22.265173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ee8a67-005d-4d94-b27f-3b1b776068de service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:22.275782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 736/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:24.454516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b11e4-0dd0-449c-9425-c4c8a885bd5e None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:24.455459 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b11e4-0dd0-449c-9425-c4c8a885bd5e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:24.455459 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b11e4-0dd0-449c-9425-c4c8a885bd5e None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:24.457395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:29:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:24.463967 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580f820-83df-4a28-b913-7773500cd2af None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:24.464768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580f820-83df-4a28-b913-7773500cd2af None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:24.464864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580f820-83df-4a28-b913-7773500cd2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:24.487038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9580f820-83df-4a28-b913-7773500cd2af None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:24.488269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9580f820-83df-4a28-b913-7773500cd2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:24.500860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:24.549399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b11e4-0dd0-449c-9425-c4c8a885bd5e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:24.561425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b11e4-0dd0-449c-9425-c4c8a885bd5e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:24.565382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f8127-cfa6-41b4-aca6-d62a86980002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:24.566021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f8127-cfa6-41b4-aca6-d62a86980002 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:24.566309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f8127-cfa6-41b4-aca6-d62a86980002 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:24.566586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f8127-cfa6-41b4-aca6-d62a86980002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:24.576519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f8127-cfa6-41b4-aca6-d62a86980002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:24.577957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f8127-cfa6-41b4-aca6-d62a86980002 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:24.588125 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 733/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:34.481359 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=11 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:29:34.583274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=14 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:29:44.452881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d17f-a1ae-410b-bfd1-aa01ab178149 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:44.453758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d17f-a1ae-410b-bfd1-aa01ab178149 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:44.453758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40d17f-a1ae-410b-bfd1-aa01ab178149 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:44.455744 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 738/1471] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:29:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:44.465356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f634b-c6bf-4537-8a5e-2fa08e36a0f3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:44.466204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f634b-c6bf-4537-8a5e-2fa08e36a0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:44.466382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f634b-c6bf-4537-8a5e-2fa08e36a0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:44.489121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-977f634b-c6bf-4537-8a5e-2fa08e36a0f3 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:44.490313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-977f634b-c6bf-4537-8a5e-2fa08e36a0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:44.500210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 734/1472] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:44.551433 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ace72f-c235-4bdb-b4ab-6e24a06553d3 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:44.551730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ace72f-c235-4bdb-b4ab-6e24a06553d3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:44.552005 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ace72f-c235-4bdb-b4ab-6e24a06553d3 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:44.553398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 739/1473] 10.222.0.27 () {54 vars in 962 bytes} [Wed Jun 29 08:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:44.558280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a0124-7fcb-46d8-a051-481d83a17b74 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:44.558565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a0124-7fcb-46d8-a051-481d83a17b74 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:44.558867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9a0124-7fcb-46d8-a051-481d83a17b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:44.577913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fd9a0124-7fcb-46d8-a051-481d83a17b74 None None] MFA Rules not processed for user `fd09453b5db645d7b03522f91effb8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:44.579127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fd9a0124-7fcb-46d8-a051-481d83a17b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:44.585827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 735/1474] 10.222.0.27 () {58 vars in 1059 bytes} [Wed Jun 29 08:29:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:44.593463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ace72f-c235-4bdb-b4ab-6e24a06553d3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:44.605545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ace72f-c235-4bdb-b4ab-6e24a06553d3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:44.609536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405bc51-51cd-489a-97d9-bd6c23e171cb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd09453b5db645d7b03522f91effb8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:44.610153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405bc51-51cd-489a-97d9-bd6c23e171cb service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:44.610432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405bc51-51cd-489a-97d9-bd6c23e171cb service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:44.610721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405bc51-51cd-489a-97d9-bd6c23e171cb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:44.620746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405bc51-51cd-489a-97d9-bd6c23e171cb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:44.622192 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405bc51-51cd-489a-97d9-bd6c23e171cb service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:44.632823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 740/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:46.818629 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6c728-924d-41eb-816a-e8be1f3b99d3 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:46.819210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6c728-924d-41eb-816a-e8be1f3b99d3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:46.819210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6c728-924d-41eb-816a-e8be1f3b99d3 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:46.820941 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 736/1476] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:46.826584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02b2e9-cb1c-4a70-8b76-b64cea50e643 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:46.827084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02b2e9-cb1c-4a70-8b76-b64cea50e643 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:46.827192 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02b2e9-cb1c-4a70-8b76-b64cea50e643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:46.847373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9b02b2e9-cb1c-4a70-8b76-b64cea50e643 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:46.848589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9b02b2e9-cb1c-4a70-8b76-b64cea50e643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:46.857541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 741/1477] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:29:46.881355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6c728-924d-41eb-816a-e8be1f3b99d3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:46.895745 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6c728-924d-41eb-816a-e8be1f3b99d3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:46.900591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6974e74f-3e6b-4eb9-b27a-7670d1ea4990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:46.901264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974e74f-3e6b-4eb9-b27a-7670d1ea4990 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:46.901551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974e74f-3e6b-4eb9-b27a-7670d1ea4990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:46.901829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6974e74f-3e6b-4eb9-b27a-7670d1ea4990 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:46.902579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974e74f-3e6b-4eb9-b27a-7670d1ea4990 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:46.904454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-6974e74f-3e6b-4eb9-b27a-7670d1ea4990 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:29:46.904454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jun 29 08:29:46.904454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: Jun 29 08:29:46.905601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6974e74f-3e6b-4eb9-b27a-7670d1ea4990 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:46.926013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 737/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Wed Jun 29 08:29:46 2022] GET /identity/v3/projects => generated 2553 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:29:48.980123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a8648-e914-4a0c-9eae-17b97a5b8742 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:48.980711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a8648-e914-4a0c-9eae-17b97a5b8742 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:48.980711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a8648-e914-4a0c-9eae-17b97a5b8742 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:48.982856 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 742/1479] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:29:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:48.991676 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89f1f-bac9-45e1-b2ae-6848464fb0ae None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:48.992315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89f1f-bac9-45e1-b2ae-6848464fb0ae None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:48.992481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89f1f-bac9-45e1-b2ae-6848464fb0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:49.017953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1cb89f1f-bac9-45e1-b2ae-6848464fb0ae None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:49.019215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1cb89f1f-bac9-45e1-b2ae-6848464fb0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:49.028781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 738/1480] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:49.072193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a8648-e914-4a0c-9eae-17b97a5b8742 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:49.087368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a8648-e914-4a0c-9eae-17b97a5b8742 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:49.094145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf654f6d-8543-4532-8b51-2bc43a47ee7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:49.094822 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf654f6d-8543-4532-8b51-2bc43a47ee7c service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:49.095110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf654f6d-8543-4532-8b51-2bc43a47ee7c service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:49.095384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf654f6d-8543-4532-8b51-2bc43a47ee7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:49.104639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf654f6d-8543-4532-8b51-2bc43a47ee7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:49.106230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf654f6d-8543-4532-8b51-2bc43a47ee7c service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:49.116308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 743/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:51.337152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bd24a-d962-416a-94f5-ab084634a342 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:51.337823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bd24a-d962-416a-94f5-ab084634a342 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:51.337823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bd24a-d962-416a-94f5-ab084634a342 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:51.339439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 739/1482] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:51.345054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d86fb4-f5d6-4ea6-9757-82a1090b3c76 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:51.345502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d86fb4-f5d6-4ea6-9757-82a1090b3c76 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:51.345636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d86fb4-f5d6-4ea6-9757-82a1090b3c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:51.364169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d1d86fb4-f5d6-4ea6-9757-82a1090b3c76 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:51.365363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d1d86fb4-f5d6-4ea6-9757-82a1090b3c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:51.373911 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 744/1483] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:51.413269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bd24a-d962-416a-94f5-ab084634a342 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:51.432029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bd24a-d962-416a-94f5-ab084634a342 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:51.436368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c36cab3-63f0-4543-993c-3fee1c4416a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:51.437048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36cab3-63f0-4543-993c-3fee1c4416a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:51.437333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36cab3-63f0-4543-993c-3fee1c4416a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:51.437610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c36cab3-63f0-4543-993c-3fee1c4416a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:51.448368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c36cab3-63f0-4543-993c-3fee1c4416a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:51.449825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c36cab3-63f0-4543-993c-3fee1c4416a5 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:51.460597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 740/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:53.607008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c021cd4-ee31-44d6-9c13-453d257ea050 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:53.607614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c021cd4-ee31-44d6-9c13-453d257ea050 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:53.607614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c021cd4-ee31-44d6-9c13-453d257ea050 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:53.608939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 745/1485] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:53.615288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24393e4-801d-4fed-bcb9-360c9d5f86e5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:53.615876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24393e4-801d-4fed-bcb9-360c9d5f86e5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:53.615876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24393e4-801d-4fed-bcb9-360c9d5f86e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:53.636885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a24393e4-801d-4fed-bcb9-360c9d5f86e5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:53.638106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a24393e4-801d-4fed-bcb9-360c9d5f86e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:53.648342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 741/1486] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:53.691973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c021cd4-ee31-44d6-9c13-453d257ea050 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:53.705708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c021cd4-ee31-44d6-9c13-453d257ea050 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:53.709553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af478d6-0777-4fe7-8cbf-02b1e0adea1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:53.710247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af478d6-0777-4fe7-8cbf-02b1e0adea1b service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:53.711113 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af478d6-0777-4fe7-8cbf-02b1e0adea1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:53.711397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af478d6-0777-4fe7-8cbf-02b1e0adea1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:53.722136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af478d6-0777-4fe7-8cbf-02b1e0adea1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:53.723545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af478d6-0777-4fe7-8cbf-02b1e0adea1b service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:53.735320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 746/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:29:55.975493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275a25e-1bf9-4660-a578-bae69dd2ef89 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:55.976071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275a25e-1bf9-4660-a578-bae69dd2ef89 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:55.976071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275a25e-1bf9-4660-a578-bae69dd2ef89 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:55.977486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 742/1488] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:55.983976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac513e-1d5b-483a-b607-97caebc1bb34 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:55.984736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac513e-1d5b-483a-b607-97caebc1bb34 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:55.984909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aac513e-1d5b-483a-b607-97caebc1bb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:56.006766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6aac513e-1d5b-483a-b607-97caebc1bb34 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:56.008196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6aac513e-1d5b-483a-b607-97caebc1bb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:56.017837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 747/1489] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:29:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:56.058481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275a25e-1bf9-4660-a578-bae69dd2ef89 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:56.070526 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275a25e-1bf9-4660-a578-bae69dd2ef89 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:56.075304 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847252c-e1aa-48c1-bbfe-8171b7faf8e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:56.076292 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847252c-e1aa-48c1-bbfe-8171b7faf8e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:56.076646 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847252c-e1aa-48c1-bbfe-8171b7faf8e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:56.077007 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847252c-e1aa-48c1-bbfe-8171b7faf8e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:56.091469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847252c-e1aa-48c1-bbfe-8171b7faf8e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:56.092902 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847252c-e1aa-48c1-bbfe-8171b7faf8e4 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:56.102904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 743/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:29:58.391240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53536e2c-a047-457e-b0b2-b9e10ac7acd8 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:58.392127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53536e2c-a047-457e-b0b2-b9e10ac7acd8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:58.392127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53536e2c-a047-457e-b0b2-b9e10ac7acd8 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:58.394335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 748/1491] 10.222.0.27 () {54 vars in 926 bytes} [Wed Jun 29 08:29:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:29:58.399699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eac456-778c-4f66-8358-a028029f1427 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:58.400152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eac456-778c-4f66-8358-a028029f1427 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:58.400248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eac456-778c-4f66-8358-a028029f1427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:58.418403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-48eac456-778c-4f66-8358-a028029f1427 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:29:58.419614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-48eac456-778c-4f66-8358-a028029f1427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:29:58.428208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 744/1492] 10.222.0.27 () {58 vars in 1023 bytes} [Wed Jun 29 08:29:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:29:58.467938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53536e2c-a047-457e-b0b2-b9e10ac7acd8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:29:58.480306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53536e2c-a047-457e-b0b2-b9e10ac7acd8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:29:58.484326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa787a-8401-4976-8a13-e923dec5fab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:29:58.484962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa787a-8401-4976-8a13-e923dec5fab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:29:58.485248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa787a-8401-4976-8a13-e923dec5fab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:29:58.485521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa787a-8401-4976-8a13-e923dec5fab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:29:58.495114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa787a-8401-4976-8a13-e923dec5fab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:29:58.496554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa787a-8401-4976-8a13-e923dec5fab6 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:29:58.506458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 749/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:30:08.413269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=33 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:30:08.503560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=33 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:31:24.875312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d2e41-b13b-4e00-9dbe-1803d0d1d647 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:24.876223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d2e41-b13b-4e00-9dbe-1803d0d1d647 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:24.876223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d2e41-b13b-4e00-9dbe-1803d0d1d647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:24.901422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f51d2e41-b13b-4e00-9dbe-1803d0d1d647 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:31:24.902646 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f51d2e41-b13b-4e00-9dbe-1803d0d1d647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:31:24.912870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 745/1494] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:31:24.925602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccd536-6681-436b-9be9-7e800d0a5780 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:24.926338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccd536-6681-436b-9be9-7e800d0a5780 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:24.926501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccd536-6681-436b-9be9-7e800d0a5780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:24.947272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-69ccd536-6681-436b-9be9-7e800d0a5780 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:31:24.948487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-69ccd536-6681-436b-9be9-7e800d0a5780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:31:24.957288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 750/1495] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:31:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:31:24.962471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d2e41-b13b-4e00-9dbe-1803d0d1d647 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:31:24.978178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d2e41-b13b-4e00-9dbe-1803d0d1d647 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:31:24.987851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff791f0-11d0-4117-8d81-3aa989598cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:31:24.988901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff791f0-11d0-4117-8d81-3aa989598cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:24.989307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff791f0-11d0-4117-8d81-3aa989598cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:24.989738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff791f0-11d0-4117-8d81-3aa989598cb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:24.991216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff791f0-11d0-4117-8d81-3aa989598cb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:31:24.991435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff791f0-11d0-4117-8d81-3aa989598cb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:31:24.992475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-5ff791f0-11d0-4117-8d81-3aa989598cb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:31:24.992475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jun 29 08:31:24.992475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: Jun 29 08:31:24.993378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff791f0-11d0-4117-8d81-3aa989598cb4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:31:25.005869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 746/1496] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:31:25.017907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccd536-6681-436b-9be9-7e800d0a5780 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:31:25.028858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccd536-6681-436b-9be9-7e800d0a5780 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:31:25.034710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a33772-aa09-4cdf-8311-17d121ee2988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:31:25.035386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a33772-aa09-4cdf-8311-17d121ee2988 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.035660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a33772-aa09-4cdf-8311-17d121ee2988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.035939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a33772-aa09-4cdf-8311-17d121ee2988 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.036639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a33772-aa09-4cdf-8311-17d121ee2988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:31:25.038566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-02a33772-aa09-4cdf-8311-17d121ee2988 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:31:25.038566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jun 29 08:31:25.038566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: Jun 29 08:31:25.039797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a33772-aa09-4cdf-8311-17d121ee2988 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:31:25.080321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 751/1497] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:31:25 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:31:25.084889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff791f0-11d0-4117-8d81-3aa989598cb4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:31:25.095662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff791f0-11d0-4117-8d81-3aa989598cb4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:31:25.156598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86458b-7d46-4851-9d38-80b414ca7f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:31:25.157389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86458b-7d46-4851-9d38-80b414ca7f2d admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.157793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86458b-7d46-4851-9d38-80b414ca7f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.158146 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86458b-7d46-4851-9d38-80b414ca7f2d admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.159266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86458b-7d46-4851-9d38-80b414ca7f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:31:25.160859 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-6d86458b-7d46-4851-9d38-80b414ca7f2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:31:25.160859 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jun 29 08:31:25.160859 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: Jun 29 08:31:25.162550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86458b-7d46-4851-9d38-80b414ca7f2d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:31:25.185266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 747/1498] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:31:25 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:31:25.189460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a33772-aa09-4cdf-8311-17d121ee2988 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:31:25.200355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a33772-aa09-4cdf-8311-17d121ee2988 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:31:25.205508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab683c78-2e4c-441d-90a2-f6f47873bf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:31:25.206204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab683c78-2e4c-441d-90a2-f6f47873bf7b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.206488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab683c78-2e4c-441d-90a2-f6f47873bf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.206794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab683c78-2e4c-441d-90a2-f6f47873bf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.207528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab683c78-2e4c-441d-90a2-f6f47873bf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:31:25.208549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-ab683c78-2e4c-441d-90a2-f6f47873bf7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:31:25.208549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jun 29 08:31:25.208549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: Jun 29 08:31:25.209399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab683c78-2e4c-441d-90a2-f6f47873bf7b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:31:25.223120 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 752/1499] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:31:25.227691 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86458b-7d46-4851-9d38-80b414ca7f2d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:31:25.237970 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86458b-7d46-4851-9d38-80b414ca7f2d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:31:25.243002 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c93f85-d820-41c0-accd-99ac46e583e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:31:25.243670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c93f85-d820-41c0-accd-99ac46e583e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.243951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c93f85-d820-41c0-accd-99ac46e583e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.244225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c93f85-d820-41c0-accd-99ac46e583e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.244980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c93f85-d820-41c0-accd-99ac46e583e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:31:25.246006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-87c93f85-d820-41c0-accd-99ac46e583e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:31:25.246006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jun 29 08:31:25.246006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: Jun 29 08:31:25.246922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c93f85-d820-41c0-accd-99ac46e583e6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:31:25.260740 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 748/1500] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:31:25.264808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab683c78-2e4c-441d-90a2-f6f47873bf7b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:31:25.275407 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab683c78-2e4c-441d-90a2-f6f47873bf7b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:31:25.280966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20748e50-d395-4d3e-8e8b-5c5b169b824e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:31:25.281849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20748e50-d395-4d3e-8e8b-5c5b169b824e admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.282130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20748e50-d395-4d3e-8e8b-5c5b169b824e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.282401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20748e50-d395-4d3e-8e8b-5c5b169b824e admin admin] PATH_INFO: `/v3/projects/9e48cf73c8e3410493716ce7524d402d/users/8c4745e9b0eb4843beb0ba1febfd1bcd/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.298173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20748e50-d395-4d3e-8e8b-5c5b169b824e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e48cf73c8e3410493716ce7524d402d, user_id=8c4745e9b0eb4843beb0ba1febfd1bcd, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:31:25.299245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-20748e50-d395-4d3e-8e8b-5c5b169b824e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:31:25.299245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jun 29 08:31:25.299245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: Jun 29 08:31:25.301164 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20748e50-d395-4d3e-8e8b-5c5b169b824e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:31:25.319418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 753/1501] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:31:25 2022] PUT /identity/v3/projects/9e48cf73c8e3410493716ce7524d402d/users/8c4745e9b0eb4843beb0ba1febfd1bcd/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:31:25.331796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0d24a-659f-43b6-ae14-cf2f853faf87 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.332082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0d24a-659f-43b6-ae14-cf2f853faf87 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.332366 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0d24a-659f-43b6-ae14-cf2f853faf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.361020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cbb0d24a-659f-43b6-ae14-cf2f853faf87 None None] MFA Rules not processed for user `8c4745e9b0eb4843beb0ba1febfd1bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:31:25.362256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cbb0d24a-659f-43b6-ae14-cf2f853faf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:31:25.434559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 749/1502] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:31:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:31:25.448202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25566245-bb1f-4dbc-b0e0-1b7ae203ef86 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.448500 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25566245-bb1f-4dbc-b0e0-1b7ae203ef86 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.448758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25566245-bb1f-4dbc-b0e0-1b7ae203ef86 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.450178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 754/1503] 10.222.0.27 () {54 vars in 963 bytes} [Wed Jun 29 08:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:31:25.455269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd6a5a-88b3-4586-8a43-eca3665c0c93 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.455560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd6a5a-88b3-4586-8a43-eca3665c0c93 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.455836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd6a5a-88b3-4586-8a43-eca3665c0c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.476692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-38bd6a5a-88b3-4586-8a43-eca3665c0c93 None None] MFA Rules not processed for user `b60ca33edf5a47fdab6e39cc98638c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:31:25.477888 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-38bd6a5a-88b3-4586-8a43-eca3665c0c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:31:25.524337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:31:25.533829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 750/1504] 10.222.0.27 () {58 vars in 1060 bytes} [Wed Jun 29 08:31:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:31:25.542447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25566245-bb1f-4dbc-b0e0-1b7ae203ef86 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:31:25.554043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25566245-bb1f-4dbc-b0e0-1b7ae203ef86 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:31:25.557940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832711ab-e7e3-4503-9781-47598d0c9958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:31:25.558589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832711ab-e7e3-4503-9781-47598d0c9958 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.558933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832711ab-e7e3-4503-9781-47598d0c9958 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.559214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832711ab-e7e3-4503-9781-47598d0c9958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.568115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832711ab-e7e3-4503-9781-47598d0c9958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:31:25.569909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832711ab-e7e3-4503-9781-47598d0c9958 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:31:25.579045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 755/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:31:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:31:25.623566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ba87f-585d-4469-bd17-6a35ba8007d9 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.623856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ba87f-585d-4469-bd17-6a35ba8007d9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.624131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ba87f-585d-4469-bd17-6a35ba8007d9 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.625440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 751/1506] 10.222.0.27 () {52 vars in 829 bytes} [Wed Jun 29 08:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:31:25.629263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832711ab-e7e3-4503-9781-47598d0c9958 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:31:25.695889 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832711ab-e7e3-4503-9781-47598d0c9958 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:31:25.702172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:31:25.703078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.703353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.703620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] PATH_INFO: `/v3/users/8c4745e9b0eb4843beb0ba1febfd1bcd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.704988 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c4745e9b0eb4843beb0ba1febfd1bcd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:31:25.706026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:31:25.706026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jun 29 08:31:25.706026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: Jun 29 08:31:25.707260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:31:25.776511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c4745e9b0eb4843beb0ba1febfd1bcd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:31:25.777178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c4745e9b0eb4843beb0ba1febfd1bcd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:31:25.797901 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c4745e9b0eb4843beb0ba1febfd1bcd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:31:25.798879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c4745e9b0eb4843beb0ba1febfd1bcd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:31:25.810842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c4745e9b0eb4843beb0ba1febfd1bcd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:31:25.819669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:31:25.831375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 756/1507] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:31:25 2022] DELETE /identity/v3/users/8c4745e9b0eb4843beb0ba1febfd1bcd => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:31:25.836251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862caea-032e-4a9c-bff2-446937a21ba1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:25.836538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862caea-032e-4a9c-bff2-446937a21ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:25.836814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862caea-032e-4a9c-bff2-446937a21ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:25.855296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3862caea-032e-4a9c-bff2-446937a21ba1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:31:25.856500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3862caea-032e-4a9c-bff2-446937a21ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:31:25.921800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 752/1508] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:31:25.933712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:31:25.935811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:31:26.000348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35577dff-05d5-4379-a3fa-31d378341b3f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:31:26.004550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f751e-1b79-4dc3-b3a7-71538c91999d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:31:26.005203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f751e-1b79-4dc3-b3a7-71538c91999d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:26.005482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f751e-1b79-4dc3-b3a7-71538c91999d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:26.005756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f751e-1b79-4dc3-b3a7-71538c91999d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:26.015512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f751e-1b79-4dc3-b3a7-71538c91999d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:31:26.016946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f751e-1b79-4dc3-b3a7-71538c91999d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:31:26.026584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 757/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:31:26.362970 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862caea-032e-4a9c-bff2-446937a21ba1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:31:26.365459 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3862caea-032e-4a9c-bff2-446937a21ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:31:26.377543 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862caea-032e-4a9c-bff2-446937a21ba1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:31:26.383043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:31:26.383924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:31:26.384201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:31:26.384472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] PATH_INFO: `/v3/projects/9e48cf73c8e3410493716ce7524d402d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:31:26.385780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e48cf73c8e3410493716ce7524d402d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:31:26.386857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:31:26.386857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jun 29 08:31:26.386857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: Jun 29 08:31:26.388063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:31:26.449595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e48cf73c8e3410493716ce7524d402d', 'request_id': 'req-46b8e3a2-0381-4196-8f6a-d66069a31888'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:31:26.463367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e48cf73c8e3410493716ce7524d402d', 'request_id': 'req-46b8e3a2-0381-4196-8f6a-d66069a31888'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:31:26.469980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e48cf73c8e3410493716ce7524d402d', 'request_id': 'req-46b8e3a2-0381-4196-8f6a-d66069a31888'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:31:26.485952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] The token cache is being invalidate because project 9e48cf73c8e3410493716ce7524d402d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:31:26.486204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:31:26.487062 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-46b8e3a2-0381-4196-8f6a-d66069a31888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:31:26.488672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 753/1510] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:31:26 2022] DELETE /identity/v3/projects/9e48cf73c8e3410493716ce7524d402d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:31:36.022485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=29 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:31:36.482827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:33:03.198566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7e132-5156-4152-b647-57a33a434057 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:03.199571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7e132-5156-4152-b647-57a33a434057 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:03.199571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7e132-5156-4152-b647-57a33a434057 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:03.201718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:33:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:33:03.209350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f2f7a-9753-4203-83b9-e438678ef994 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:03.209827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f2f7a-9753-4203-83b9-e438678ef994 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:03.209931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f2f7a-9753-4203-83b9-e438678ef994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:03.239654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a33f2f7a-9753-4203-83b9-e438678ef994 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:33:03.240861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a33f2f7a-9753-4203-83b9-e438678ef994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:03.306442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:33:03.461200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7e132-5156-4152-b647-57a33a434057 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:33:03.463398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6ec7e132-5156-4152-b647-57a33a434057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:03.546428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7e132-5156-4152-b647-57a33a434057 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:33:03.552751 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb83af96-8000-483b-a5fa-449b5557fceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:33:03.553396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83af96-8000-483b-a5fa-449b5557fceb service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:03.553675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83af96-8000-483b-a5fa-449b5557fceb service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:03.553952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb83af96-8000-483b-a5fa-449b5557fceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:03.562655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83af96-8000-483b-a5fa-449b5557fceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:33:03.564287 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb83af96-8000-483b-a5fa-449b5557fceb service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:33:03.573899 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 759/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:33:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:33:04.410666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aead2c5-925d-4392-99af-dae44c0f88eb None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:04.411607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aead2c5-925d-4392-99af-dae44c0f88eb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:04.411607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aead2c5-925d-4392-99af-dae44c0f88eb None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:04.413848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:33:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:33:04.419972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286f3c8-74e8-4b7b-8207-cb280c8ded50 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:04.420766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286f3c8-74e8-4b7b-8207-cb280c8ded50 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:04.420951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286f3c8-74e8-4b7b-8207-cb280c8ded50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:04.447320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9286f3c8-74e8-4b7b-8207-cb280c8ded50 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:33:04.448546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9286f3c8-74e8-4b7b-8207-cb280c8ded50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:04.458322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:33:04.613624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aead2c5-925d-4392-99af-dae44c0f88eb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:33:04.638995 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aead2c5-925d-4392-99af-dae44c0f88eb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:33:04.646928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e873e1-d65d-496d-ba51-b2971180c1c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:33:04.647583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e873e1-d65d-496d-ba51-b2971180c1c6 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:04.647862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e873e1-d65d-496d-ba51-b2971180c1c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:04.648132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e873e1-d65d-496d-ba51-b2971180c1c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:04.657129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e873e1-d65d-496d-ba51-b2971180c1c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:33:04.658965 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e873e1-d65d-496d-ba51-b2971180c1c6 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:33:04.669390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 756/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:33:04.923976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdc03-8c62-4da4-ba49-5036a990c787 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:04.924216 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdc03-8c62-4da4-ba49-5036a990c787 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:04.924444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdc03-8c62-4da4-ba49-5036a990c787 None None] PATH_INFO: `/v3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:04.926209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 761/1517] 10.222.0.52 () {54 vars in 925 bytes} [Wed Jun 29 08:33:04 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:33:04.933726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567d1fa-4e6f-410f-9b6d-22cd25a0ab79 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:04.934021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567d1fa-4e6f-410f-9b6d-22cd25a0ab79 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:04.934290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567d1fa-4e6f-410f-9b6d-22cd25a0ab79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:04.954885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8567d1fa-4e6f-410f-9b6d-22cd25a0ab79 None None] MFA Rules not processed for user `00a08f31638d4eaab5ca83d6f493db6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:33:04.956127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8567d1fa-4e6f-410f-9b6d-22cd25a0ab79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:05.033807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 757/1518] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Jun 29 08:33:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:33:14.111892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113f7e8-a613-4544-ad3f-92e6cffcfd96 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:14.112450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113f7e8-a613-4544-ad3f-92e6cffcfd96 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:14.112450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113f7e8-a613-4544-ad3f-92e6cffcfd96 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:14.113769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 762/1519] 10.222.0.27 () {54 vars in 960 bytes} [Wed Jun 29 08:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:33:14.120055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae1c45-5ce1-4834-af52-dec2dac259f3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:14.120808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae1c45-5ce1-4834-af52-dec2dac259f3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:14.120944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae1c45-5ce1-4834-af52-dec2dac259f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:14.144307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-92ae1c45-5ce1-4834-af52-dec2dac259f3 None None] MFA Rules not processed for user `00a08f31638d4eaab5ca83d6f493db6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:33:14.145991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-92ae1c45-5ce1-4834-af52-dec2dac259f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:14.155564 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 758/1520] 10.222.0.27 () {58 vars in 1057 bytes} [Wed Jun 29 08:33:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:33:14.163256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113f7e8-a613-4544-ad3f-92e6cffcfd96 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:33:14.177209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113f7e8-a613-4544-ad3f-92e6cffcfd96 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:33:14.182615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c931da-540a-46f7-a15f-5d97d524cf35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:33:14.183269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c931da-540a-46f7-a15f-5d97d524cf35 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:14.183546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c931da-540a-46f7-a15f-5d97d524cf35 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:14.183823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c931da-540a-46f7-a15f-5d97d524cf35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:14.192243 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c931da-540a-46f7-a15f-5d97d524cf35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:33:14.193819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c931da-540a-46f7-a15f-5d97d524cf35 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:33:14.203390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 763/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:33:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:33:15.924718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfaf59-6641-42e5-8dbe-4b883b1050dc None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:15.925384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfaf59-6641-42e5-8dbe-4b883b1050dc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:15.925384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfaf59-6641-42e5-8dbe-4b883b1050dc None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:15.926786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:33:15.932809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42376ba3-b77b-4565-9b7d-ab26afef173a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:15.933282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42376ba3-b77b-4565-9b7d-ab26afef173a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:15.933386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42376ba3-b77b-4565-9b7d-ab26afef173a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:15.959048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-42376ba3-b77b-4565-9b7d-ab26afef173a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:33:15.960261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-42376ba3-b77b-4565-9b7d-ab26afef173a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:15.968890 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:33:16.124224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfaf59-6641-42e5-8dbe-4b883b1050dc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:33:16.141325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfaf59-6641-42e5-8dbe-4b883b1050dc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:33:16.147522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bf6c4-5682-431f-9a0c-46958ad65ec8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:33:16.148168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bf6c4-5682-431f-9a0c-46958ad65ec8 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:16.148450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bf6c4-5682-431f-9a0c-46958ad65ec8 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:16.148722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bf6c4-5682-431f-9a0c-46958ad65ec8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:16.157359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7bf6c4-5682-431f-9a0c-46958ad65ec8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:33:16.158984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7bf6c4-5682-431f-9a0c-46958ad65ec8 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:33:16.168778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 760/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:33:16.989053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729022a0-e4c1-452b-bcdb-8be2408a160f None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:16.989970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729022a0-e4c1-452b-bcdb-8be2408a160f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:16.990151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729022a0-e4c1-452b-bcdb-8be2408a160f None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:16.992548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 08:33:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:33:16.998493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907857ce-8013-4389-a606-26c2ff2cae00 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:16.999251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907857ce-8013-4389-a606-26c2ff2cae00 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:16.999526 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907857ce-8013-4389-a606-26c2ff2cae00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:17.020286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-907857ce-8013-4389-a606-26c2ff2cae00 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:33:17.021478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-907857ce-8013-4389-a606-26c2ff2cae00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:17.030296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 08:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:33:17.185678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729022a0-e4c1-452b-bcdb-8be2408a160f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:33:17.204753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729022a0-e4c1-452b-bcdb-8be2408a160f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:33:17.211496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edd035-3c32-4f84-af4c-b44211e5f5e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:33:17.212134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edd035-3c32-4f84-af4c-b44211e5f5e5 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:17.212415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edd035-3c32-4f84-af4c-b44211e5f5e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:17.212690 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edd035-3c32-4f84-af4c-b44211e5f5e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:17.221259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edd035-3c32-4f84-af4c-b44211e5f5e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:33:17.223038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edd035-3c32-4f84-af4c-b44211e5f5e5 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:33:17.232512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 766/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:33:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:33:21.226855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9689052-b055-4094-b53d-bac3ca9f3c30 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:21.227463 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9689052-b055-4094-b53d-bac3ca9f3c30 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:21.227463 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9689052-b055-4094-b53d-bac3ca9f3c30 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:21.229455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:33:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:33:21.238660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97810f-2799-43d3-8469-2d1af39b6ef6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:21.239158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97810f-2799-43d3-8469-2d1af39b6ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:21.239362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97810f-2799-43d3-8469-2d1af39b6ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:21.266728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5c97810f-2799-43d3-8469-2d1af39b6ef6 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:33:21.268457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5c97810f-2799-43d3-8469-2d1af39b6ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:21.278308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:33:21.318970 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9689052-b055-4094-b53d-bac3ca9f3c30 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:33:21.321297 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e9689052-b055-4094-b53d-bac3ca9f3c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:21.384927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9689052-b055-4094-b53d-bac3ca9f3c30 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:33:21.388895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538712c-100f-4a47-aa4c-c3bc4b417897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:33:21.389547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538712c-100f-4a47-aa4c-c3bc4b417897 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:21.389828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538712c-100f-4a47-aa4c-c3bc4b417897 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:21.390113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538712c-100f-4a47-aa4c-c3bc4b417897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:21.399831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538712c-100f-4a47-aa4c-c3bc4b417897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:33:21.401272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538712c-100f-4a47-aa4c-c3bc4b417897 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:33:21.406432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:33:21.412032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 763/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:33:23.446941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcad158-688f-4ad2-9ce9-60153cfb3423 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:23.447859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcad158-688f-4ad2-9ce9-60153cfb3423 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:23.447859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcad158-688f-4ad2-9ce9-60153cfb3423 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:23.450034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:33:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:33:23.455635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f049e-aa44-4537-aee9-efd600728067 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:23.456138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f049e-aa44-4537-aee9-efd600728067 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:23.456138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f049e-aa44-4537-aee9-efd600728067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:23.476737 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-aa6f049e-aa44-4537-aee9-efd600728067 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:33:23.477922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-aa6f049e-aa44-4537-aee9-efd600728067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:23.487109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:33:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:33:23.525297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcad158-688f-4ad2-9ce9-60153cfb3423 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:33:23.527660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1fcad158-688f-4ad2-9ce9-60153cfb3423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:23.545540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcad158-688f-4ad2-9ce9-60153cfb3423 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:33:23.551946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b510e-d342-4714-b4ad-613f60b07728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:33:23.552628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b510e-d342-4714-b4ad-613f60b07728 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:23.553002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b510e-d342-4714-b4ad-613f60b07728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:23.553273 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b510e-d342-4714-b4ad-613f60b07728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:23.562989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b510e-d342-4714-b4ad-613f60b07728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:33:23.564441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b510e-d342-4714-b4ad-613f60b07728 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:33:23.574298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 769/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:33:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:33:27.125944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a319-d9a1-41dc-aafa-3982e15e427c None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:27.126946 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a319-d9a1-41dc-aafa-3982e15e427c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:27.126946 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a319-d9a1-41dc-aafa-3982e15e427c None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:27.129231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:33:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:33:27.135382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710a3b6-fa78-41ac-a653-04df4f14b582 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:27.136136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710a3b6-fa78-41ac-a653-04df4f14b582 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:27.136272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710a3b6-fa78-41ac-a653-04df4f14b582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:27.146284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=62 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:33:27.165900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f710a3b6-fa78-41ac-a653-04df4f14b582 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:33:27.167450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f710a3b6-fa78-41ac-a653-04df4f14b582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:27.179341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:33:27.495303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a319-d9a1-41dc-aafa-3982e15e427c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:33:27.497228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7894a319-d9a1-41dc-aafa-3982e15e427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:27.558246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894a319-d9a1-41dc-aafa-3982e15e427c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:33:27.563427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c2e24-6266-4bd3-b1f3-ca91f88de5f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:33:27.564072 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c2e24-6266-4bd3-b1f3-ca91f88de5f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:27.564360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c2e24-6266-4bd3-b1f3-ca91f88de5f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:27.564634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c2e24-6266-4bd3-b1f3-ca91f88de5f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:27.574777 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c2e24-6266-4bd3-b1f3-ca91f88de5f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:33:27.576254 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c2e24-6266-4bd3-b1f3-ca91f88de5f9 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:33:27.587249 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 766/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:33:29.642563 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcd498-0900-436a-a3e0-9b629013863d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:29.643501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcd498-0900-436a-a3e0-9b629013863d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:29.643501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcd498-0900-436a-a3e0-9b629013863d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:29.646437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 08:33:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:33:29.655572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb1377-c196-42fb-95f2-57b61e15a90f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:29.656001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb1377-c196-42fb-95f2-57b61e15a90f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:29.656124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb1377-c196-42fb-95f2-57b61e15a90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:29.677061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3acb1377-c196-42fb-95f2-57b61e15a90f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:33:29.678263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3acb1377-c196-42fb-95f2-57b61e15a90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:29.687710 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 08:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:33:29.987747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcd498-0900-436a-a3e0-9b629013863d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:33:29.990068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9ffcd498-0900-436a-a3e0-9b629013863d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:33:30.003498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcd498-0900-436a-a3e0-9b629013863d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:33:30.007528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f7084-9120-471b-bad5-1dd93268aab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:33:30.008171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f7084-9120-471b-bad5-1dd93268aab9 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:33:30.008459 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f7084-9120-471b-bad5-1dd93268aab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:33:30.008734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f7084-9120-471b-bad5-1dd93268aab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:33:30.017911 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f7084-9120-471b-bad5-1dd93268aab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:33:30.019397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f7084-9120-471b-bad5-1dd93268aab9 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:33:30.029013 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 772/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:33:39.671754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=32 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:33:40.024759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=8 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:34:59.789322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edf027-e2f1-4b58-a6a3-ad8443ec2175 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:34:59.790909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edf027-e2f1-4b58-a6a3-ad8443ec2175 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:34:59.790909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edf027-e2f1-4b58-a6a3-ad8443ec2175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:34:59.793023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037dc02-8166-4f38-946d-3ad65d21969a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:34:59.793465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037dc02-8166-4f38-946d-3ad65d21969a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:34:59.793560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037dc02-8166-4f38-946d-3ad65d21969a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:34:59.817234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a4edf027-e2f1-4b58-a6a3-ad8443ec2175 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:34:59.819397 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a4edf027-e2f1-4b58-a6a3-ad8443ec2175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:34:59.826840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4037dc02-8166-4f38-946d-3ad65d21969a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:34:59.828079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4037dc02-8166-4f38-946d-3ad65d21969a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:34:59.833103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 768/1540] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:34:59.834419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3ea4e-ee41-4b66-a6b9-a0782141cd47 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:34:59.834721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3ea4e-ee41-4b66-a6b9-a0782141cd47 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:34:59.835009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3ea4e-ee41-4b66-a6b9-a0782141cd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:34:59.850233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 773/1541] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:34:59.859542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f63355-e5df-4f4e-85c3-a5ef00ae04cc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:34:59.859869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f63355-e5df-4f4e-85c3-a5ef00ae04cc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:34:59.860085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f63355-e5df-4f4e-85c3-a5ef00ae04cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:34:59.863507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-70b3ea4e-ee41-4b66-a6b9-a0782141cd47 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:34:59.865149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-70b3ea4e-ee41-4b66-a6b9-a0782141cd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:34:59.878534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 769/1542] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:34:59.884157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8deca-ed96-4761-b140-b701e1b92f27 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:34:59.884467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8deca-ed96-4761-b140-b701e1b92f27 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:34:59.884725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8deca-ed96-4761-b140-b701e1b92f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:34:59.919749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9fb8deca-ed96-4761-b140-b701e1b92f27 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:34:59.921178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9fb8deca-ed96-4761-b140-b701e1b92f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:34:59.952028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 770/1543] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:34:59.953584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95ed71-044c-4d1c-82a1-8792272739d0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:34:59.953877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95ed71-044c-4d1c-82a1-8792272739d0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:34:59.954156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95ed71-044c-4d1c-82a1-8792272739d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:34:59.977065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-df95ed71-044c-4d1c-82a1-8792272739d0 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:34:59.978872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-df95ed71-044c-4d1c-82a1-8792272739d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:34:59.987657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 771/1544] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:34:59.989642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae3398-dae1-420b-b0cc-6ac3fb7cc1cb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:34:59.989932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae3398-dae1-420b-b0cc-6ac3fb7cc1cb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:34:59.990266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae3398-dae1-420b-b0cc-6ac3fb7cc1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.033823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d2ae3398-dae1-420b-b0cc-6ac3fb7cc1cb None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:00.035276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d2ae3398-dae1-420b-b0cc-6ac3fb7cc1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:00.035420 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-02f63355-e5df-4f4e-85c3-a5ef00ae04cc None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:00.036693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-02f63355-e5df-4f4e-85c3-a5ef00ae04cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:00.044208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 772/1545] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:00.048071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 774/1546] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:35:00.048071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f280f94-6ea7-4bfe-a730-bbeb06a7bf08 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.048071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f280f94-6ea7-4bfe-a730-bbeb06a7bf08 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.048071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f280f94-6ea7-4bfe-a730-bbeb06a7bf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.052733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3805-fb82-4d57-adce-964c55208f0c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.053277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3805-fb82-4d57-adce-964c55208f0c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.054152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3805-fb82-4d57-adce-964c55208f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.065991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9f280f94-6ea7-4bfe-a730-bbeb06a7bf08 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:00.067269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9f280f94-6ea7-4bfe-a730-bbeb06a7bf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:00.075244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 775/1547] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:00.076051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f280f94-6ea7-4bfe-a730-bbeb06a7bf08 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.077231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ea4a3805-fb82-4d57-adce-964c55208f0c None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:00.078597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ea4a3805-fb82-4d57-adce-964c55208f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:00.085608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f280f94-6ea7-4bfe-a730-bbeb06a7bf08 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.091172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05057c2-6cd1-487f-a20b-aa060f825fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.091829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05057c2-6cd1-487f-a20b-aa060f825fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.092105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05057c2-6cd1-487f-a20b-aa060f825fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.092405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05057c2-6cd1-487f-a20b-aa060f825fd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.093132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05057c2-6cd1-487f-a20b-aa060f825fd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:00.093433 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05057c2-6cd1-487f-a20b-aa060f825fd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.094451 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-b05057c2-6cd1-487f-a20b-aa060f825fd2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:35:00.094451 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jun 29 08:35:00.094451 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: Jun 29 08:35:00.095256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 773/1548] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:00.095392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05057c2-6cd1-487f-a20b-aa060f825fd2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.096131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a3805-fb82-4d57-adce-964c55208f0c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.109593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a3805-fb82-4d57-adce-964c55208f0c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.118437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 776/1549] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:00.118437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb4bc9-7bd4-47fe-8d84-bcd9cd8dc3b5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.118437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb4bc9-7bd4-47fe-8d84-bcd9cd8dc3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.118437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb4bc9-7bd4-47fe-8d84-bcd9cd8dc3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.118850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958dd69-d0c8-4aef-ae8f-1cfc9127fccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.118850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958dd69-d0c8-4aef-ae8f-1cfc9127fccd admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.118850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958dd69-d0c8-4aef-ae8f-1cfc9127fccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.118850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958dd69-d0c8-4aef-ae8f-1cfc9127fccd admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.119156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958dd69-d0c8-4aef-ae8f-1cfc9127fccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:00.119156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958dd69-d0c8-4aef-ae8f-1cfc9127fccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.119735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d958dd69-d0c8-4aef-ae8f-1cfc9127fccd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.135142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 774/1550] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:00.136214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958dd69-d0c8-4aef-ae8f-1cfc9127fccd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.138891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e8fb4bc9-7bd4-47fe-8d84-bcd9cd8dc3b5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:00.140088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e8fb4bc9-7bd4-47fe-8d84-bcd9cd8dc3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:00.146919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958dd69-d0c8-4aef-ae8f-1cfc9127fccd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.151259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 777/1551] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:00.151633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c65d4a-3d85-4bfd-9795-cd759ae315d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.152604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c65d4a-3d85-4bfd-9795-cd759ae315d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.152923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c65d4a-3d85-4bfd-9795-cd759ae315d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.153026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721d523-862b-4833-9427-3ac785a602d4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.153168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c65d4a-3d85-4bfd-9795-cd759ae315d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.153440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721d523-862b-4833-9427-3ac785a602d4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.153746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721d523-862b-4833-9427-3ac785a602d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.153906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c65d4a-3d85-4bfd-9795-cd759ae315d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:00.154226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c65d4a-3d85-4bfd-9795-cd759ae315d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.158983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c65d4a-3d85-4bfd-9795-cd759ae315d6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.172979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 775/1552] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:00.174495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06d123-e90f-455b-bb49-5015f887db1d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.174812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06d123-e90f-455b-bb49-5015f887db1d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.175090 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06d123-e90f-455b-bb49-5015f887db1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.177160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c721d523-862b-4833-9427-3ac785a602d4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:00.178888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c721d523-862b-4833-9427-3ac785a602d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:00.192965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 778/1553] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:00.193597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0c06d123-e90f-455b-bb49-5015f887db1d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:00.194578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98937c07-122d-4ff4-aad5-b11a11fa4f99 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.194896 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98937c07-122d-4ff4-aad5-b11a11fa4f99 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.195012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0c06d123-e90f-455b-bb49-5015f887db1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:00.195206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98937c07-122d-4ff4-aad5-b11a11fa4f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.203617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 776/1554] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:00.204856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf502b8-259c-479e-bbaa-0f2b3b3bec76 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.205142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf502b8-259c-479e-bbaa-0f2b3b3bec76 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.205380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf502b8-259c-479e-bbaa-0f2b3b3bec76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.217002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-98937c07-122d-4ff4-aad5-b11a11fa4f99 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:00.218219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-98937c07-122d-4ff4-aad5-b11a11fa4f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:00.224268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-acf502b8-259c-479e-bbaa-0f2b3b3bec76 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:00.225404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-acf502b8-259c-479e-bbaa-0f2b3b3bec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:00.227983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 779/1555] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:00.229456 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98937c07-122d-4ff4-aad5-b11a11fa4f99 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.233842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 777/1556] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:00.234983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf502b8-259c-479e-bbaa-0f2b3b3bec76 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.241603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98937c07-122d-4ff4-aad5-b11a11fa4f99 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.245589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf502b8-259c-479e-bbaa-0f2b3b3bec76 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.246484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee6a7a-02e6-4c55-ad6e-79a9741de425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.247196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee6a7a-02e6-4c55-ad6e-79a9741de425 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.247470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee6a7a-02e6-4c55-ad6e-79a9741de425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.247745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee6a7a-02e6-4c55-ad6e-79a9741de425 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.248452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee6a7a-02e6-4c55-ad6e-79a9741de425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.250547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee6a7a-02e6-4c55-ad6e-79a9741de425 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.251293 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b82ca-4de1-48fd-91c4-aff685ab2c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.252071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b82ca-4de1-48fd-91c4-aff685ab2c84 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.252621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b82ca-4de1-48fd-91c4-aff685ab2c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.252940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b82ca-4de1-48fd-91c4-aff685ab2c84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.253851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b82ca-4de1-48fd-91c4-aff685ab2c84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.255411 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-6b7b82ca-4de1-48fd-91c4-aff685ab2c84 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:35:00.255411 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jun 29 08:35:00.255411 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: Jun 29 08:35:00.257149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b82ca-4de1-48fd-91c4-aff685ab2c84 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.283045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 780/1557] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:00.284524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f02f1a-45f3-40f3-b8be-176d7904127f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.284930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f02f1a-45f3-40f3-b8be-176d7904127f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.285271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f02f1a-45f3-40f3-b8be-176d7904127f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.290739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 778/1558] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:00.290739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b82ca-4de1-48fd-91c4-aff685ab2c84 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.307368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e9f02f1a-45f3-40f3-b8be-176d7904127f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:00.308493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e9f02f1a-45f3-40f3-b8be-176d7904127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:00.359869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b82ca-4de1-48fd-91c4-aff685ab2c84 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.365273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6067f-0f79-45cd-a9da-43278a2c9f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.365957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6067f-0f79-45cd-a9da-43278a2c9f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.366243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6067f-0f79-45cd-a9da-43278a2c9f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.366522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6067f-0f79-45cd-a9da-43278a2c9f93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.367254 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6067f-0f79-45cd-a9da-43278a2c9f93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.369222 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6067f-0f79-45cd-a9da-43278a2c9f93 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.373747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 781/1559] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:00.374622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f02f1a-45f3-40f3-b8be-176d7904127f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.386412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f02f1a-45f3-40f3-b8be-176d7904127f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.393766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccea3b0-6f59-4b37-8dd4-5c2d5757c1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.395095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccea3b0-6f59-4b37-8dd4-5c2d5757c1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.395414 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccea3b0-6f59-4b37-8dd4-5c2d5757c1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.395855 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccea3b0-6f59-4b37-8dd4-5c2d5757c1e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.396625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccea3b0-6f59-4b37-8dd4-5c2d5757c1e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:00.396925 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccea3b0-6f59-4b37-8dd4-5c2d5757c1e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.398537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccea3b0-6f59-4b37-8dd4-5c2d5757c1e4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.400507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 779/1560] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:00.402313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6067f-0f79-45cd-a9da-43278a2c9f93 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.412775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 782/1561] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:00.414099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbb823-f7e1-4f72-99e0-b53febbf35fb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.414385 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbb823-f7e1-4f72-99e0-b53febbf35fb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.414639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbb823-f7e1-4f72-99e0-b53febbf35fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.433974 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fcfbb823-f7e1-4f72-99e0-b53febbf35fb None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:00.435210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fcfbb823-f7e1-4f72-99e0-b53febbf35fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:00.460638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:00.479798 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6067f-0f79-45cd-a9da-43278a2c9f93 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.486148 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a277f-5265-4b1b-8bc9-20ddd8a74868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.486908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a277f-5265-4b1b-8bc9-20ddd8a74868 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.487193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a277f-5265-4b1b-8bc9-20ddd8a74868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.487658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a277f-5265-4b1b-8bc9-20ddd8a74868 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.488560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a277f-5265-4b1b-8bc9-20ddd8a74868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:00.489041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a277f-5265-4b1b-8bc9-20ddd8a74868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.490716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a277f-5265-4b1b-8bc9-20ddd8a74868 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.503010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 780/1562] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:00.504076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a277f-5265-4b1b-8bc9-20ddd8a74868 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.507946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 783/1563] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:00.508576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbb823-f7e1-4f72-99e0-b53febbf35fb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.514091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a277f-5265-4b1b-8bc9-20ddd8a74868 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.520074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cb339-0b73-4ca6-884a-a66b5d7f9a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.520787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb339-0b73-4ca6-884a-a66b5d7f9a3c admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.521065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb339-0b73-4ca6-884a-a66b5d7f9a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.521344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb339-0b73-4ca6-884a-a66b5d7f9a3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.522059 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbb823-f7e1-4f72-99e0-b53febbf35fb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.522214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cb339-0b73-4ca6-884a-a66b5d7f9a3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:00.522411 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cb339-0b73-4ca6-884a-a66b5d7f9a3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.524137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cb339-0b73-4ca6-884a-a66b5d7f9a3c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.527918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a51ed8-0354-4111-b84a-841d8488a73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.528965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51ed8-0354-4111-b84a-841d8488a73c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.529248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51ed8-0354-4111-b84a-841d8488a73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.529518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51ed8-0354-4111-b84a-841d8488a73c admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.530219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a51ed8-0354-4111-b84a-841d8488a73c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.532208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-07a51ed8-0354-4111-b84a-841d8488a73c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:35:00.532208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jun 29 08:35:00.532208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: Jun 29 08:35:00.533423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a51ed8-0354-4111-b84a-841d8488a73c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.537715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 781/1564] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:00.538633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cb339-0b73-4ca6-884a-a66b5d7f9a3c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.545169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:00.551396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cb339-0b73-4ca6-884a-a66b5d7f9a3c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.557789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6983fde-0ce2-423a-895a-2ede56b1eb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.558479 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983fde-0ce2-423a-895a-2ede56b1eb93 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.559023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983fde-0ce2-423a-895a-2ede56b1eb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.559542 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983fde-0ce2-423a-895a-2ede56b1eb93 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.560452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6983fde-0ce2-423a-895a-2ede56b1eb93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.561335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 784/1565] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/users => generated 289 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:00.562191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a51ed8-0354-4111-b84a-841d8488a73c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.562894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6983fde-0ce2-423a-895a-2ede56b1eb93 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.574618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a51ed8-0354-4111-b84a-841d8488a73c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.580599 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dc19b-a943-452f-a96d-c4c55eb81549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.581378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc19b-a943-452f-a96d-c4c55eb81549 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.581880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc19b-a943-452f-a96d-c4c55eb81549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.582484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dc19b-a943-452f-a96d-c4c55eb81549 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.582867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dc19b-a943-452f-a96d-c4c55eb81549 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:00.583127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dc19b-a943-452f-a96d-c4c55eb81549 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.585179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dc19b-a943-452f-a96d-c4c55eb81549 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.592725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 782/1566] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/users => generated 287 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:00.592725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6983fde-0ce2-423a-895a-2ede56b1eb93 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.601697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 785/1567] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:00.601922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6983fde-0ce2-423a-895a-2ede56b1eb93 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.602727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dc19b-a943-452f-a96d-c4c55eb81549 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.614621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d255a3-352c-4260-845d-1f34b04fa567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.615440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d255a3-352c-4260-845d-1f34b04fa567 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.615814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d255a3-352c-4260-845d-1f34b04fa567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.616325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d255a3-352c-4260-845d-1f34b04fa567 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.617095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d255a3-352c-4260-845d-1f34b04fa567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.619755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d255a3-352c-4260-845d-1f34b04fa567 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.620713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dc19b-a943-452f-a96d-c4c55eb81549 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.625990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c4722-12a8-477a-8e41-e954ee139355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.626719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c4722-12a8-477a-8e41-e954ee139355 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.626979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c4722-12a8-477a-8e41-e954ee139355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.627255 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c4722-12a8-477a-8e41-e954ee139355 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.627986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c4722-12a8-477a-8e41-e954ee139355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.629871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c4722-12a8-477a-8e41-e954ee139355 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.646586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 783/1568] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/users => generated 287 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:00.646586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d255a3-352c-4260-845d-1f34b04fa567 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.655511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d255a3-352c-4260-845d-1f34b04fa567 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.660830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84eb076-e492-4df3-b989-b686d38a62e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.661644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eb076-e492-4df3-b989-b686d38a62e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.661934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eb076-e492-4df3-b989-b686d38a62e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.662207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84eb076-e492-4df3-b989-b686d38a62e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.662936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84eb076-e492-4df3-b989-b686d38a62e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.664887 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84eb076-e492-4df3-b989-b686d38a62e0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.668551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 786/1569] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:00.669710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c4722-12a8-477a-8e41-e954ee139355 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.699908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 784/1570] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:00.700828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84eb076-e492-4df3-b989-b686d38a62e0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.746368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c4722-12a8-477a-8e41-e954ee139355 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.772229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84eb076-e492-4df3-b989-b686d38a62e0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.777142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d3865-a824-430b-b09f-d71bee7f3f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.777830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d3865-a824-430b-b09f-d71bee7f3f66 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.778109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d3865-a824-430b-b09f-d71bee7f3f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.778361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d3865-a824-430b-b09f-d71bee7f3f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.779080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d3865-a824-430b-b09f-d71bee7f3f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.781010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d3865-a824-430b-b09f-d71bee7f3f66 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.814304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974afd85-28fe-48ec-9b1d-e405307e8fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.815122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974afd85-28fe-48ec-9b1d-e405307e8fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.815236 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974afd85-28fe-48ec-9b1d-e405307e8fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.815567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974afd85-28fe-48ec-9b1d-e405307e8fbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.816402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974afd85-28fe-48ec-9b1d-e405307e8fbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:00.816709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974afd85-28fe-48ec-9b1d-e405307e8fbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.818338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974afd85-28fe-48ec-9b1d-e405307e8fbe admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.818860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 785/1571] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:00.820085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d3865-a824-430b-b09f-d71bee7f3f66 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.840149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 787/1572] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:00.840149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974afd85-28fe-48ec-9b1d-e405307e8fbe admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.847553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974afd85-28fe-48ec-9b1d-e405307e8fbe admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.879382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:00.892153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d3865-a824-430b-b09f-d71bee7f3f66 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.897833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be650d4-ac77-4a5e-8d91-0789b5cb5f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.899026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be650d4-ac77-4a5e-8d91-0789b5cb5f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.899497 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be650d4-ac77-4a5e-8d91-0789b5cb5f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.899779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be650d4-ac77-4a5e-8d91-0789b5cb5f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.900679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be650d4-ac77-4a5e-8d91-0789b5cb5f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.903267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be650d4-ac77-4a5e-8d91-0789b5cb5f67 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.914019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f144bb-968a-4a74-9525-8946b35d6b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.914258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 786/1573] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:00.914883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f144bb-968a-4a74-9525-8946b35d6b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.915151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f144bb-968a-4a74-9525-8946b35d6b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.915570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f144bb-968a-4a74-9525-8946b35d6b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.916324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f144bb-968a-4a74-9525-8946b35d6b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.917685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be650d4-ac77-4a5e-8d91-0789b5cb5f67 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.918373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f144bb-968a-4a74-9525-8946b35d6b3a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.921624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:00.928350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be650d4-ac77-4a5e-8d91-0789b5cb5f67 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.933410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676cbba-aa03-48d2-8bfb-d17e60014858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.933717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 788/1574] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:00.934081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676cbba-aa03-48d2-8bfb-d17e60014858 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.934366 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676cbba-aa03-48d2-8bfb-d17e60014858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.934641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6676cbba-aa03-48d2-8bfb-d17e60014858 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.935738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676cbba-aa03-48d2-8bfb-d17e60014858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.936730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f144bb-968a-4a74-9525-8946b35d6b3a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.937400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6676cbba-aa03-48d2-8bfb-d17e60014858 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.951321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f144bb-968a-4a74-9525-8946b35d6b3a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:00.958423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630da2b-99fa-4678-bcb7-ea2895be7b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:00.959530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630da2b-99fa-4678-bcb7-ea2895be7b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:00.959812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630da2b-99fa-4678-bcb7-ea2895be7b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:00.960100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630da2b-99fa-4678-bcb7-ea2895be7b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:00.961403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630da2b-99fa-4678-bcb7-ea2895be7b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:00.963731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630da2b-99fa-4678-bcb7-ea2895be7b12 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:00.972664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 787/1575] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:00.972664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676cbba-aa03-48d2-8bfb-d17e60014858 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.978529 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 789/1576] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:00.979277 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630da2b-99fa-4678-bcb7-ea2895be7b12 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:00.989050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630da2b-99fa-4678-bcb7-ea2895be7b12 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.047923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6676cbba-aa03-48d2-8bfb-d17e60014858 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.053135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31615e8-7aa3-47a5-a576-810b2ee91086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.053817 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31615e8-7aa3-47a5-a576-810b2ee91086 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.054098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31615e8-7aa3-47a5-a576-810b2ee91086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.054372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31615e8-7aa3-47a5-a576-810b2ee91086 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.055093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31615e8-7aa3-47a5-a576-810b2ee91086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.056634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab995ed-f747-46e6-9ab0-14bb5b694837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.057167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31615e8-7aa3-47a5-a576-810b2ee91086 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.057319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab995ed-f747-46e6-9ab0-14bb5b694837 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.057605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab995ed-f747-46e6-9ab0-14bb5b694837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.058086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab995ed-f747-46e6-9ab0-14bb5b694837 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.058963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab995ed-f747-46e6-9ab0-14bb5b694837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.061122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab995ed-f747-46e6-9ab0-14bb5b694837 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.082413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 788/1577] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:01.083179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31615e8-7aa3-47a5-a576-810b2ee91086 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.087400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 790/1578] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:00 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:01.087400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab995ed-f747-46e6-9ab0-14bb5b694837 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.093637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31615e8-7aa3-47a5-a576-810b2ee91086 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.097517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab995ed-f747-46e6-9ab0-14bb5b694837 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.098979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ad5cd-a23c-4d86-b663-56dacb594c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.099657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ad5cd-a23c-4d86-b663-56dacb594c31 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.099934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ad5cd-a23c-4d86-b663-56dacb594c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.100198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ad5cd-a23c-4d86-b663-56dacb594c31 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.100902 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ad5cd-a23c-4d86-b663-56dacb594c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.101974 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be69e2-c0c4-4ac2-8f78-76bd656e436d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.102619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be69e2-c0c4-4ac2-8f78-76bd656e436d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.102853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ad5cd-a23c-4d86-b663-56dacb594c31 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.102959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be69e2-c0c4-4ac2-8f78-76bd656e436d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.103193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be69e2-c0c4-4ac2-8f78-76bd656e436d admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.103873 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be69e2-c0c4-4ac2-8f78-76bd656e436d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.105746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be69e2-c0c4-4ac2-8f78-76bd656e436d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.123833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 789/1579] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:01 2022] POST /identity/v3/users => generated 294 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:01.124813 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ad5cd-a23c-4d86-b663-56dacb594c31 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.133918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 791/1580] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:01 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:01.134769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be69e2-c0c4-4ac2-8f78-76bd656e436d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.137835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ad5cd-a23c-4d86-b663-56dacb594c31 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.195559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b6b41-ec88-48c6-b05a-be6f51c449d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.196246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b6b41-ec88-48c6-b05a-be6f51c449d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.196533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b6b41-ec88-48c6-b05a-be6f51c449d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.196811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b6b41-ec88-48c6-b05a-be6f51c449d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.197582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b6b41-ec88-48c6-b05a-be6f51c449d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.197699 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be69e2-c0c4-4ac2-8f78-76bd656e436d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.199336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b6b41-ec88-48c6-b05a-be6f51c449d7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.202103 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82132e-dc4e-4ab9-aea5-2fc382b12a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.202801 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82132e-dc4e-4ab9-aea5-2fc382b12a74 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.203083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82132e-dc4e-4ab9-aea5-2fc382b12a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.203353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82132e-dc4e-4ab9-aea5-2fc382b12a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.204112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82132e-dc4e-4ab9-aea5-2fc382b12a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.205697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82132e-dc4e-4ab9-aea5-2fc382b12a74 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.209559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 790/1581] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.210441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b6b41-ec88-48c6-b05a-be6f51c449d7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.216305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 792/1582] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.217123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82132e-dc4e-4ab9-aea5-2fc382b12a74 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.224231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b6b41-ec88-48c6-b05a-be6f51c449d7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.227018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82132e-dc4e-4ab9-aea5-2fc382b12a74 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.229754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc9c3-ee89-48bf-bda8-d6846228840e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.230428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc9c3-ee89-48bf-bda8-d6846228840e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.230708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc9c3-ee89-48bf-bda8-d6846228840e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.231003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc9c3-ee89-48bf-bda8-d6846228840e admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.231690 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafc9c3-ee89-48bf-bda8-d6846228840e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.233419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd27227-285b-4117-be08-09936661d74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.233602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafc9c3-ee89-48bf-bda8-d6846228840e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.234073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd27227-285b-4117-be08-09936661d74c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.234358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd27227-285b-4117-be08-09936661d74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.234638 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd27227-285b-4117-be08-09936661d74c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.235431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd27227-285b-4117-be08-09936661d74c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.237014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd27227-285b-4117-be08-09936661d74c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.247898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 793/1583] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.248971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd27227-285b-4117-be08-09936661d74c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.258778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 791/1584] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:01 2022] POST /identity/v3/users => generated 283 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:01.259371 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd27227-285b-4117-be08-09936661d74c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.259951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc9c3-ee89-48bf-bda8-d6846228840e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.265524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafccb47-bf3f-4293-ac30-3ba5b35bbb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.266192 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafccb47-bf3f-4293-ac30-3ba5b35bbb45 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.266489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafccb47-bf3f-4293-ac30-3ba5b35bbb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.266781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafccb47-bf3f-4293-ac30-3ba5b35bbb45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.267542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafccb47-bf3f-4293-ac30-3ba5b35bbb45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.269232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafccb47-bf3f-4293-ac30-3ba5b35bbb45 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.269789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc9c3-ee89-48bf-bda8-d6846228840e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.275396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde64d4-acee-45f7-92f9-bcf338ff7f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.276059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde64d4-acee-45f7-92f9-bcf338ff7f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.276344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde64d4-acee-45f7-92f9-bcf338ff7f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.276623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde64d4-acee-45f7-92f9-bcf338ff7f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.277356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde64d4-acee-45f7-92f9-bcf338ff7f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.278978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde64d4-acee-45f7-92f9-bcf338ff7f39 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.280241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 794/1585] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.281061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafccb47-bf3f-4293-ac30-3ba5b35bbb45 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.291141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafccb47-bf3f-4293-ac30-3ba5b35bbb45 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.292839 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 792/1586] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.293648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde64d4-acee-45f7-92f9-bcf338ff7f39 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.296004 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72646544-67e5-4d97-9325-ab0a4649b1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.296660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72646544-67e5-4d97-9325-ab0a4649b1bd admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.296928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72646544-67e5-4d97-9325-ab0a4649b1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.297201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72646544-67e5-4d97-9325-ab0a4649b1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.297921 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72646544-67e5-4d97-9325-ab0a4649b1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.299569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72646544-67e5-4d97-9325-ab0a4649b1bd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.303683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde64d4-acee-45f7-92f9-bcf338ff7f39 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.308471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b86a5-1197-4997-b2ad-658ec5cd7954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.309181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b86a5-1197-4997-b2ad-658ec5cd7954 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.309468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b86a5-1197-4997-b2ad-658ec5cd7954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.309749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b86a5-1197-4997-b2ad-658ec5cd7954 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.310606 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b86a5-1197-4997-b2ad-658ec5cd7954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.312603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b86a5-1197-4997-b2ad-658ec5cd7954 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.314042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 795/1587] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.315304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72646544-67e5-4d97-9325-ab0a4649b1bd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.320030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:01.328397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72646544-67e5-4d97-9325-ab0a4649b1bd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.334227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98e919-b20b-48c9-aa58-6e54ab5c5cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.335206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98e919-b20b-48c9-aa58-6e54ab5c5cd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.335487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98e919-b20b-48c9-aa58-6e54ab5c5cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.335779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98e919-b20b-48c9-aa58-6e54ab5c5cd2 admin admin] PATH_INFO: `/v3/projects/ed50fef9e1824a2c918d9649f226364d/users/d55c898180bd4cbf910f3d92045e7d67/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.336257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 793/1588] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:01 2022] POST /identity/v3/users => generated 291 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:01.337114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b86a5-1197-4997-b2ad-658ec5cd7954 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.347605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b86a5-1197-4997-b2ad-658ec5cd7954 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.351829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98e919-b20b-48c9-aa58-6e54ab5c5cd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed50fef9e1824a2c918d9649f226364d, user_id=d55c898180bd4cbf910f3d92045e7d67, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.352926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39674a-cf57-43eb-8c7c-29d47ab2e988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.353831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39674a-cf57-43eb-8c7c-29d47ab2e988 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.354110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39674a-cf57-43eb-8c7c-29d47ab2e988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.354389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39674a-cf57-43eb-8c7c-29d47ab2e988 admin admin] PATH_INFO: `/v3/projects/3682cd9a64cd469684178faa933eb5dd/users/7d0db1e0495a4cb1ad076985f0cf03e8/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.354705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98e919-b20b-48c9-aa58-6e54ab5c5cd2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.357075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:01.370639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 796/1589] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:01 2022] PUT /identity/v3/projects/ed50fef9e1824a2c918d9649f226364d/users/d55c898180bd4cbf910f3d92045e7d67/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:01.374981 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98e919-b20b-48c9-aa58-6e54ab5c5cd2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.375096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39674a-cf57-43eb-8c7c-29d47ab2e988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3682cd9a64cd469684178faa933eb5dd, user_id=7d0db1e0495a4cb1ad076985f0cf03e8, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.375096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-7f39674a-cf57-43eb-8c7c-29d47ab2e988 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:35:01.375096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jun 29 08:35:01.375096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: Jun 29 08:35:01.375096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39674a-cf57-43eb-8c7c-29d47ab2e988 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.390840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 794/1590] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:01 2022] PUT /identity/v3/projects/3682cd9a64cd469684178faa933eb5dd/users/7d0db1e0495a4cb1ad076985f0cf03e8/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:01.391838 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39674a-cf57-43eb-8c7c-29d47ab2e988 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.444402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98e919-b20b-48c9-aa58-6e54ab5c5cd2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.458424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39674a-cf57-43eb-8c7c-29d47ab2e988 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.463495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b2272-80bd-498d-b925-ebaf2da54c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.464299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b2272-80bd-498d-b925-ebaf2da54c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.465004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b2272-80bd-498d-b925-ebaf2da54c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.465283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14b2272-80bd-498d-b925-ebaf2da54c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.466028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b2272-80bd-498d-b925-ebaf2da54c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.467693 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14b2272-80bd-498d-b925-ebaf2da54c4a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.478639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 795/1591] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.479737 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b2272-80bd-498d-b925-ebaf2da54c4a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.491187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14b2272-80bd-498d-b925-ebaf2da54c4a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.495879 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebeb2b-f8f6-4914-9072-a51e61c670d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.496547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebeb2b-f8f6-4914-9072-a51e61c670d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.496837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebeb2b-f8f6-4914-9072-a51e61c670d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.497116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebeb2b-f8f6-4914-9072-a51e61c670d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.497847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebeb2b-f8f6-4914-9072-a51e61c670d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.499498 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebeb2b-f8f6-4914-9072-a51e61c670d4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.506370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd1aa9-c76b-42b0-8cd3-1e64a4a63348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.507305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd1aa9-c76b-42b0-8cd3-1e64a4a63348 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.507587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd1aa9-c76b-42b0-8cd3-1e64a4a63348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.507864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd1aa9-c76b-42b0-8cd3-1e64a4a63348 admin admin] PATH_INFO: `/v3/projects/4e1c22413fe7410ab520a8891c1e0f65/users/370e9262843b4ba7a907886fd6e08ac5/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.512833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 796/1592] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.512833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebeb2b-f8f6-4914-9072-a51e61c670d4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.519831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebeb2b-f8f6-4914-9072-a51e61c670d4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.522984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd1aa9-c76b-42b0-8cd3-1e64a4a63348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e1c22413fe7410ab520a8891c1e0f65, user_id=370e9262843b4ba7a907886fd6e08ac5, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.524672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a2784-04d6-44e9-b9b0-18c57620a03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.525337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a2784-04d6-44e9-b9b0-18c57620a03c admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.525623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a2784-04d6-44e9-b9b0-18c57620a03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.525848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd1aa9-c76b-42b0-8cd3-1e64a4a63348 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.525940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a2784-04d6-44e9-b9b0-18c57620a03c admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.526625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a2784-04d6-44e9-b9b0-18c57620a03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.528258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a2784-04d6-44e9-b9b0-18c57620a03c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.539883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 797/1593] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.540681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a2784-04d6-44e9-b9b0-18c57620a03c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.540923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 797/1594] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:01 2022] PUT /identity/v3/projects/4e1c22413fe7410ab520a8891c1e0f65/users/370e9262843b4ba7a907886fd6e08ac5/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:01.541736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd1aa9-c76b-42b0-8cd3-1e64a4a63348 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.561728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a2784-04d6-44e9-b9b0-18c57620a03c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.601497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd1aa9-c76b-42b0-8cd3-1e64a4a63348 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.606124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904939e7-1c07-4382-8eed-e40adcdeac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.606954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904939e7-1c07-4382-8eed-e40adcdeac49 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.607346 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904939e7-1c07-4382-8eed-e40adcdeac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.607625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904939e7-1c07-4382-8eed-e40adcdeac49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.608354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904939e7-1c07-4382-8eed-e40adcdeac49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.609961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904939e7-1c07-4382-8eed-e40adcdeac49 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.618278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1d0b2-d5ea-4908-96be-634198c47aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.619003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1d0b2-d5ea-4908-96be-634198c47aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.619291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1d0b2-d5ea-4908-96be-634198c47aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.619568 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1d0b2-d5ea-4908-96be-634198c47aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.620176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 798/1595] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.620311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1d0b2-d5ea-4908-96be-634198c47aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.621482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd50040-346c-41aa-bfaa-c8d6a6aa516a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.621772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd50040-346c-41aa-bfaa-c8d6a6aa516a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.621882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1d0b2-d5ea-4908-96be-634198c47aa7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.622056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd50040-346c-41aa-bfaa-c8d6a6aa516a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.632834 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 798/1596] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.633975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bee83f-0105-4b61-b015-07db1ebe4d2e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.634267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bee83f-0105-4b61-b015-07db1ebe4d2e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.634543 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bee83f-0105-4b61-b015-07db1ebe4d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.642807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1dd50040-346c-41aa-bfaa-c8d6a6aa516a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:01.644000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1dd50040-346c-41aa-bfaa-c8d6a6aa516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:01.652399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 799/1597] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:01.653125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd50040-346c-41aa-bfaa-c8d6a6aa516a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.663476 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd50040-346c-41aa-bfaa-c8d6a6aa516a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.668935 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3441cc-c522-4afd-8f28-6adfef59a25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.669220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e2bee83f-0105-4b61-b015-07db1ebe4d2e None None] MFA Rules not processed for user `7d0db1e0495a4cb1ad076985f0cf03e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:01.669601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3441cc-c522-4afd-8f28-6adfef59a25d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.669878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3441cc-c522-4afd-8f28-6adfef59a25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.670147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3441cc-c522-4afd-8f28-6adfef59a25d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.670438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e2bee83f-0105-4b61-b015-07db1ebe4d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:01.670909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3441cc-c522-4afd-8f28-6adfef59a25d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.672512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3441cc-c522-4afd-8f28-6adfef59a25d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.683548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 800/1598] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.684121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3441cc-c522-4afd-8f28-6adfef59a25d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.689948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:01.694102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3441cc-c522-4afd-8f28-6adfef59a25d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.696783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:01.699879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2355f-6bef-490f-8723-f02f9b5f63b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.700918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2355f-6bef-490f-8723-f02f9b5f63b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.701394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2355f-6bef-490f-8723-f02f9b5f63b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.701670 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2355f-6bef-490f-8723-f02f9b5f63b5 admin admin] PATH_INFO: `/v3/projects/d7d568938e284206a571e256a12cac66/users/75171062556f41e5af6b5b1c4d5da7b6/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.718003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2355f-6bef-490f-8723-f02f9b5f63b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d568938e284206a571e256a12cac66, user_id=75171062556f41e5af6b5b1c4d5da7b6, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.720654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2355f-6bef-490f-8723-f02f9b5f63b5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.736114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 801/1599] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:01 2022] PUT /identity/v3/projects/d7d568938e284206a571e256a12cac66/users/75171062556f41e5af6b5b1c4d5da7b6/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:01.747075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2355f-6bef-490f-8723-f02f9b5f63b5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.747190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 799/1600] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:01.747190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6045f4a-c5b3-485b-86c1-083fc27b5df8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.747190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6045f4a-c5b3-485b-86c1-083fc27b5df8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.747190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6045f4a-c5b3-485b-86c1-083fc27b5df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.769849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c6045f4a-c5b3-485b-86c1-083fc27b5df8 None None] MFA Rules not processed for user `370e9262843b4ba7a907886fd6e08ac5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:01.771142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c6045f4a-c5b3-485b-86c1-083fc27b5df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:01.805854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2355f-6bef-490f-8723-f02f9b5f63b5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.811471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebbd25-d5e1-40e0-926c-3ba0419912a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.812382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebbd25-d5e1-40e0-926c-3ba0419912a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.812666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebbd25-d5e1-40e0-926c-3ba0419912a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.812939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebbd25-d5e1-40e0-926c-3ba0419912a4 admin admin] PATH_INFO: `/v3/projects/96872c14d1224c92b3247285f03bb966/users/ac984d1f6efa403e85483bd7fa78000e/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.828649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebbd25-d5e1-40e0-926c-3ba0419912a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96872c14d1224c92b3247285f03bb966, user_id=ac984d1f6efa403e85483bd7fa78000e, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.831345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebbd25-d5e1-40e0-926c-3ba0419912a4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.835142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 800/1601] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:01.840329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6045f4a-c5b3-485b-86c1-083fc27b5df8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.848756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6045f4a-c5b3-485b-86c1-083fc27b5df8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.852869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 802/1602] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:01 2022] PUT /identity/v3/projects/96872c14d1224c92b3247285f03bb966/users/ac984d1f6efa403e85483bd7fa78000e/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:01.853687 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebbd25-d5e1-40e0-926c-3ba0419912a4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.911325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161c81a-26dc-4b45-9cdc-c82301a88c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.912008 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161c81a-26dc-4b45-9cdc-c82301a88c9d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.912293 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161c81a-26dc-4b45-9cdc-c82301a88c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.912569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161c81a-26dc-4b45-9cdc-c82301a88c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.913303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161c81a-26dc-4b45-9cdc-c82301a88c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.914952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b161c81a-26dc-4b45-9cdc-c82301a88c9d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.920767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebbd25-d5e1-40e0-926c-3ba0419912a4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.925119 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 801/1603] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:01.925591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f54a-201f-42bf-9816-ccb05c517547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:01.926214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161c81a-26dc-4b45-9cdc-c82301a88c9d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.926512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f54a-201f-42bf-9816-ccb05c517547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:01.926848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f54a-201f-42bf-9816-ccb05c517547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:01.927118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f54a-201f-42bf-9816-ccb05c517547 admin admin] PATH_INFO: `/v3/projects/c3addc279ffb4a49b379a938b1fbed3c/users/37d6c5d1f76445499566472213f13e3a/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:01.943209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4f54a-201f-42bf-9816-ccb05c517547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3addc279ffb4a49b379a938b1fbed3c, user_id=37d6c5d1f76445499566472213f13e3a, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:01.945907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4f54a-201f-42bf-9816-ccb05c517547 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:01.963866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 803/1604] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:01 2022] PUT /identity/v3/projects/c3addc279ffb4a49b379a938b1fbed3c/users/37d6c5d1f76445499566472213f13e3a/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:01.963866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f54a-201f-42bf-9816-ccb05c517547 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:01.982960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161c81a-26dc-4b45-9cdc-c82301a88c9d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:01.986599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:02.008496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:02.026869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f54a-201f-42bf-9816-ccb05c517547 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.031643 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad797fa-3cce-40d0-ab17-78c78123af02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.032598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad797fa-3cce-40d0-ab17-78c78123af02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.032826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad797fa-3cce-40d0-ab17-78c78123af02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.033102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad797fa-3cce-40d0-ab17-78c78123af02 admin admin] PATH_INFO: `/v3/projects/38f7be8ca176417c8edd2b5f778c08c5/users/a1c9e67668b74cb49a124da506ee03b5/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.033212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbed79-da90-454d-8874-b296e4467491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.033860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbed79-da90-454d-8874-b296e4467491 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.034143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbed79-da90-454d-8874-b296e4467491 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.034420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbed79-da90-454d-8874-b296e4467491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.044011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbed79-da90-454d-8874-b296e4467491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.045489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbed79-da90-454d-8874-b296e4467491 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.047508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad797fa-3cce-40d0-ab17-78c78123af02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f7be8ca176417c8edd2b5f778c08c5, user_id=a1c9e67668b74cb49a124da506ee03b5, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.050250 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad797fa-3cce-40d0-ab17-78c78123af02 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.055868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 802/1605] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:02.056917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbed79-da90-454d-8874-b296e4467491 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.065347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 804/1606] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:01 2022] PUT /identity/v3/projects/38f7be8ca176417c8edd2b5f778c08c5/users/a1c9e67668b74cb49a124da506ee03b5/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:02.066627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997b28-d0d3-4269-bff0-793f15e02446 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.067035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997b28-d0d3-4269-bff0-793f15e02446 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.067318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997b28-d0d3-4269-bff0-793f15e02446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.072510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbed79-da90-454d-8874-b296e4467491 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.096237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-17997b28-d0d3-4269-bff0-793f15e02446 None None] MFA Rules not processed for user `75171062556f41e5af6b5b1c4d5da7b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:02.098265 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-17997b28-d0d3-4269-bff0-793f15e02446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:02.140368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc96a5-1768-4b92-8606-f7cf882b929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.141032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc96a5-1768-4b92-8606-f7cf882b929e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.141331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc96a5-1768-4b92-8606-f7cf882b929e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.141630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc96a5-1768-4b92-8606-f7cf882b929e admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.142484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc96a5-1768-4b92-8606-f7cf882b929e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.144624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc96a5-1768-4b92-8606-f7cf882b929e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.174840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 803/1607] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:02.176628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b0376-67ca-4ed7-8bb1-f563e5b10b95 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.176981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b0376-67ca-4ed7-8bb1-f563e5b10b95 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.177366 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b0376-67ca-4ed7-8bb1-f563e5b10b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.178835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 805/1608] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:02.182313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327998b-79e0-4a52-82f5-711b0086237d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.182606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327998b-79e0-4a52-82f5-711b0086237d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.182906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327998b-79e0-4a52-82f5-711b0086237d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.184417 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 806/1609] 10.222.0.27 () {54 vars in 961 bytes} [Wed Jun 29 08:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:35:02.184919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327998b-79e0-4a52-82f5-711b0086237d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.202312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c04b0376-67ca-4ed7-8bb1-f563e5b10b95 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:02.203581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c04b0376-67ca-4ed7-8bb1-f563e5b10b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:02.266374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327998b-79e0-4a52-82f5-711b0086237d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.271456 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb77a54-7c3b-4fc5-973d-a8951748d9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.272368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb77a54-7c3b-4fc5-973d-a8951748d9b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.272650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb77a54-7c3b-4fc5-973d-a8951748d9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.272926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb77a54-7c3b-4fc5-973d-a8951748d9b8 admin admin] PATH_INFO: `/v3/projects/8f94b91ae9ee464aa2f5e24f5e8e0f7b/users/bccf002d9da34d1dad0ee71b5b7bb1c1/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.278284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 804/1610] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:02.279907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f67b23-3ea9-4089-abe9-08dd97f2d134 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.280171 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f67b23-3ea9-4089-abe9-08dd97f2d134 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.280450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f67b23-3ea9-4089-abe9-08dd97f2d134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.292662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb77a54-7c3b-4fc5-973d-a8951748d9b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f94b91ae9ee464aa2f5e24f5e8e0f7b, user_id=bccf002d9da34d1dad0ee71b5b7bb1c1, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.295893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb77a54-7c3b-4fc5-973d-a8951748d9b8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.300272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-54f67b23-3ea9-4089-abe9-08dd97f2d134 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:02.301755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-54f67b23-3ea9-4089-abe9-08dd97f2d134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:02.314205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 805/1611] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:02.314205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619ac4-f983-47b9-b458-601c94dd763b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.314205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619ac4-f983-47b9-b458-601c94dd763b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.314205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d619ac4-f983-47b9-b458-601c94dd763b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.314584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 807/1612] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:02 2022] PUT /identity/v3/projects/8f94b91ae9ee464aa2f5e24f5e8e0f7b/users/bccf002d9da34d1dad0ee71b5b7bb1c1/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:02.316157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb77a54-7c3b-4fc5-973d-a8951748d9b8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.349413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0d619ac4-f983-47b9-b458-601c94dd763b None None] MFA Rules not processed for user `a1c9e67668b74cb49a124da506ee03b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:02.350952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0d619ac4-f983-47b9-b458-601c94dd763b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:02.361262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:02.373872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:02.386730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb77a54-7c3b-4fc5-973d-a8951748d9b8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.391738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20c6e6-7795-400b-b1f8-c03bb64ae913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.392532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20c6e6-7795-400b-b1f8-c03bb64ae913 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.392832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20c6e6-7795-400b-b1f8-c03bb64ae913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.393112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20c6e6-7795-400b-b1f8-c03bb64ae913 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.393806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20c6e6-7795-400b-b1f8-c03bb64ae913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.395816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20c6e6-7795-400b-b1f8-c03bb64ae913 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.424252 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 808/1613] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:02.425161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20c6e6-7795-400b-b1f8-c03bb64ae913 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.435070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20c6e6-7795-400b-b1f8-c03bb64ae913 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.436371 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 806/1614] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:02.437867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b9447-bf50-470b-848c-f9d918b7cdfd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.438157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b9447-bf50-470b-848c-f9d918b7cdfd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.438432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b9447-bf50-470b-848c-f9d918b7cdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.443556 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2853aa-e7e4-4f0d-b73b-a3d887eceb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.444232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2853aa-e7e4-4f0d-b73b-a3d887eceb9d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.444509 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2853aa-e7e4-4f0d-b73b-a3d887eceb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.444874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2853aa-e7e4-4f0d-b73b-a3d887eceb9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.445515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2853aa-e7e4-4f0d-b73b-a3d887eceb9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.447517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2853aa-e7e4-4f0d-b73b-a3d887eceb9d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.457649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-090b9447-bf50-470b-848c-f9d918b7cdfd None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:02.458900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-090b9447-bf50-470b-848c-f9d918b7cdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:02.490773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 809/1615] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:02.495386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2853aa-e7e4-4f0d-b73b-a3d887eceb9d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.548783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 807/1616] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:02.550391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88282a8-96c5-42f1-97b2-cedb38caa63f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.551023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88282a8-96c5-42f1-97b2-cedb38caa63f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.551535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88282a8-96c5-42f1-97b2-cedb38caa63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.567204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2853aa-e7e4-4f0d-b73b-a3d887eceb9d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.571234 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2ce9f-50b2-47ef-b851-1b336913855a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.572063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ce9f-50b2-47ef-b851-1b336913855a service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.572345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ce9f-50b2-47ef-b851-1b336913855a service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.572795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2ce9f-50b2-47ef-b851-1b336913855a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.573611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e88282a8-96c5-42f1-97b2-cedb38caa63f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:02.574799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e88282a8-96c5-42f1-97b2-cedb38caa63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:02.644742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 808/1617] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:02.647482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2ce9f-50b2-47ef-b851-1b336913855a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.648607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88282a8-96c5-42f1-97b2-cedb38caa63f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.649262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2ce9f-50b2-47ef-b851-1b336913855a service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.658419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88282a8-96c5-42f1-97b2-cedb38caa63f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.661341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 810/1618] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:02.662395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2ce9f-50b2-47ef-b851-1b336913855a service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.662852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649734c-e556-45ac-9141-cf9bceb749a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.663585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649734c-e556-45ac-9141-cf9bceb749a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.663933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649734c-e556-45ac-9141-cf9bceb749a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.664296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4649734c-e556-45ac-9141-cf9bceb749a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.671621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2ce9f-50b2-47ef-b851-1b336913855a service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.675581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649734c-e556-45ac-9141-cf9bceb749a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.677293 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4649734c-e556-45ac-9141-cf9bceb749a6 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.678177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68cb77-e77a-46dc-8f75-f2f9a44dc69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.678879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68cb77-e77a-46dc-8f75-f2f9a44dc69c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.679160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68cb77-e77a-46dc-8f75-f2f9a44dc69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.679430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68cb77-e77a-46dc-8f75-f2f9a44dc69c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.680154 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68cb77-e77a-46dc-8f75-f2f9a44dc69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.681855 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68cb77-e77a-46dc-8f75-f2f9a44dc69c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.689809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 809/1619] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:02.690842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649734c-e556-45ac-9141-cf9bceb749a6 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.696664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 811/1620] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:02.696664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68cb77-e77a-46dc-8f75-f2f9a44dc69c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.702893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4649734c-e556-45ac-9141-cf9bceb749a6 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.708027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68cb77-e77a-46dc-8f75-f2f9a44dc69c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.708721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48ef2b-9c9e-4f7b-81de-d293341b9787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.709401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ef2b-9c9e-4f7b-81de-d293341b9787 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.709781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ef2b-9c9e-4f7b-81de-d293341b9787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.710087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ef2b-9c9e-4f7b-81de-d293341b9787 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.711196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48ef2b-9c9e-4f7b-81de-d293341b9787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.712904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd48ef2b-9c9e-4f7b-81de-d293341b9787 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.713249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451587c-937e-4f1e-9165-f8cfd7000104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.714259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451587c-937e-4f1e-9165-f8cfd7000104 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.715534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451587c-937e-4f1e-9165-f8cfd7000104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.716005 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451587c-937e-4f1e-9165-f8cfd7000104 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.716878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451587c-937e-4f1e-9165-f8cfd7000104 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.719485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451587c-937e-4f1e-9165-f8cfd7000104 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.743105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 810/1621] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:02.744168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48ef2b-9c9e-4f7b-81de-d293341b9787 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.749888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 812/1622] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/users => generated 293 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:02.749888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451587c-937e-4f1e-9165-f8cfd7000104 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.761949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451587c-937e-4f1e-9165-f8cfd7000104 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.767272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:02.794523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:02.828124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858493e-179a-41c0-91a0-74da57450644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.828685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48ef2b-9c9e-4f7b-81de-d293341b9787 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.829203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858493e-179a-41c0-91a0-74da57450644 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.829677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858493e-179a-41c0-91a0-74da57450644 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.829778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858493e-179a-41c0-91a0-74da57450644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.841780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8fc6d-934c-4eac-9120-634c1b44baa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.843465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8fc6d-934c-4eac-9120-634c1b44baa6 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.843968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8fc6d-934c-4eac-9120-634c1b44baa6 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.844260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8fc6d-934c-4eac-9120-634c1b44baa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.917377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8fc6d-934c-4eac-9120-634c1b44baa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.919265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8fc6d-934c-4eac-9120-634c1b44baa6 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.919853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858493e-179a-41c0-91a0-74da57450644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.921345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858493e-179a-41c0-91a0-74da57450644 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.929796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 811/1623] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:02.930551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8fc6d-934c-4eac-9120-634c1b44baa6 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.933762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 813/1624] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:02.934900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858493e-179a-41c0-91a0-74da57450644 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:02.954658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858493e-179a-41c0-91a0-74da57450644 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.962276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8fc6d-934c-4eac-9120-634c1b44baa6 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:02.965489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b600e-e157-42ee-88f3-3e1827981e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.967043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b600e-e157-42ee-88f3-3e1827981e97 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.967411 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b600e-e157-42ee-88f3-3e1827981e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.968357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b600e-e157-42ee-88f3-3e1827981e97 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:02.969666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b600e-e157-42ee-88f3-3e1827981e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:02.971718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477e97b-6691-43d7-a104-e2349132becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:02.972856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477e97b-6691-43d7-a104-e2349132becc admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:02.973399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477e97b-6691-43d7-a104-e2349132becc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:02.973581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b600e-e157-42ee-88f3-3e1827981e97 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:02.973936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477e97b-6691-43d7-a104-e2349132becc admin admin] PATH_INFO: `/v3/projects/d4c43fbf2a64457da1df8ad944af013b/users/f7196aa1e6ee4809a8a2a15d62cc7053/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.012256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477e97b-6691-43d7-a104-e2349132becc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c43fbf2a64457da1df8ad944af013b, user_id=f7196aa1e6ee4809a8a2a15d62cc7053, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:03.017626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477e97b-6691-43d7-a104-e2349132becc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:03.059780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 814/1625] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:02 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:03.062233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b600e-e157-42ee-88f3-3e1827981e97 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:03.065149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 812/1626] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:02 2022] PUT /identity/v3/projects/d4c43fbf2a64457da1df8ad944af013b/users/f7196aa1e6ee4809a8a2a15d62cc7053/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:03.066358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477e97b-6691-43d7-a104-e2349132becc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:03.077093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b600e-e157-42ee-88f3-3e1827981e97 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:03.174684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a638f-717c-42f9-ab92-b85163bb6820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:03.176447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a638f-717c-42f9-ab92-b85163bb6820 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.176762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a638f-717c-42f9-ab92-b85163bb6820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.177048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a638f-717c-42f9-ab92-b85163bb6820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.177855 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a638f-717c-42f9-ab92-b85163bb6820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:03.180704 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a638f-717c-42f9-ab92-b85163bb6820 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:03.199802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 815/1627] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:03.200886 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a638f-717c-42f9-ab92-b85163bb6820 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:03.212400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a638f-717c-42f9-ab92-b85163bb6820 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:03.222449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99e3fa-5319-4cd2-af9b-79ac1aaa85cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:03.223259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e3fa-5319-4cd2-af9b-79ac1aaa85cf admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.223455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e3fa-5319-4cd2-af9b-79ac1aaa85cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.223770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99e3fa-5319-4cd2-af9b-79ac1aaa85cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.224520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99e3fa-5319-4cd2-af9b-79ac1aaa85cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:03.228604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477e97b-6691-43d7-a104-e2349132becc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:03.228919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99e3fa-5319-4cd2-af9b-79ac1aaa85cf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:03.240479 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed9c76-8bbb-450e-989f-cd0cf65ca47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:03.241173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed9c76-8bbb-450e-989f-cd0cf65ca47a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.241455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed9c76-8bbb-450e-989f-cd0cf65ca47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.241998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed9c76-8bbb-450e-989f-cd0cf65ca47a admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.242789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed9c76-8bbb-450e-989f-cd0cf65ca47a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:03.245183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed9c76-8bbb-450e-989f-cd0cf65ca47a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:03.247375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 816/1628] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:03.248380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99e3fa-5319-4cd2-af9b-79ac1aaa85cf admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:03.261237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99e3fa-5319-4cd2-af9b-79ac1aaa85cf admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:03.263853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 813/1629] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:03.270550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940a118-bfb4-438c-9dba-2c0bef6a5158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:03.271612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc451e-1407-4e18-af8c-287509024480 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.271796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940a118-bfb4-438c-9dba-2c0bef6a5158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.272206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940a118-bfb4-438c-9dba-2c0bef6a5158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.272321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc451e-1407-4e18-af8c-287509024480 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.272735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc451e-1407-4e18-af8c-287509024480 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.272883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940a118-bfb4-438c-9dba-2c0bef6a5158 admin admin] PATH_INFO: `/v3/projects/605016267c164bda8174a225d8943f76/users/1755834ded10440495f0e7a790a45339/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.275080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 814/1630] 10.222.0.27 () {54 vars in 912 bytes} [Wed Jun 29 08:35:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:35:03.276343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc451e-1407-4e18-af8c-287509024480 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:03.297544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc451e-1407-4e18-af8c-287509024480 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:03.302524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940a118-bfb4-438c-9dba-2c0bef6a5158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=605016267c164bda8174a225d8943f76, user_id=1755834ded10440495f0e7a790a45339, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:03.306372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f47f0-501d-4fb0-b37e-d91126844d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:03.307508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f47f0-501d-4fb0-b37e-d91126844d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.307940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f47f0-501d-4fb0-b37e-d91126844d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.308508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f47f0-501d-4fb0-b37e-d91126844d10 admin admin] PATH_INFO: `/v3/projects/d4c43fbf2a64457da1df8ad944af013b/users/f7196aa1e6ee4809a8a2a15d62cc7053/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.309122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8940a118-bfb4-438c-9dba-2c0bef6a5158 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:03.312063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f47f0-501d-4fb0-b37e-d91126844d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c43fbf2a64457da1df8ad944af013b, user_id=f7196aa1e6ee4809a8a2a15d62cc7053, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:03.316671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f47f0-501d-4fb0-b37e-d91126844d10 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:03.336967 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 817/1631] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:03 2022] PUT /identity/v3/projects/605016267c164bda8174a225d8943f76/users/1755834ded10440495f0e7a790a45339/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:03.341444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940a118-bfb4-438c-9dba-2c0bef6a5158 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:03.366769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 815/1632] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:03 2022] PUT /identity/v3/projects/d4c43fbf2a64457da1df8ad944af013b/users/f7196aa1e6ee4809a8a2a15d62cc7053/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:03.366769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef16275-2d89-45c7-9484-c1dde78e8039 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.366769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef16275-2d89-45c7-9484-c1dde78e8039 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.366769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef16275-2d89-45c7-9484-c1dde78e8039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.393494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:03.398223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3ef16275-2d89-45c7-9484-c1dde78e8039 None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:03.401926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3ef16275-2d89-45c7-9484-c1dde78e8039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:03.417051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:03.475344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8940a118-bfb4-438c-9dba-2c0bef6a5158 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:03.565524 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 816/1633] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:35:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:03.571913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef16275-2d89-45c7-9484-c1dde78e8039 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:03.617031 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b5a1f-95a8-4651-be1d-58ee264261f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:03.618264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5a1f-95a8-4651-be1d-58ee264261f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.619517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5a1f-95a8-4651-be1d-58ee264261f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.619996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5a1f-95a8-4651-be1d-58ee264261f2 admin admin] PATH_INFO: `/v3/projects/99b4b8d0934f4c2e9c9d5b92d144eb09/users/3433365739164957b4a911a699e32479/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.645568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b5a1f-95a8-4651-be1d-58ee264261f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b4b8d0934f4c2e9c9d5b92d144eb09, user_id=3433365739164957b4a911a699e32479, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:03.649598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b5a1f-95a8-4651-be1d-58ee264261f2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:03.684424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 818/1634] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:03 2022] PUT /identity/v3/projects/99b4b8d0934f4c2e9c9d5b92d144eb09/users/3433365739164957b4a911a699e32479/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:03.687352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abdd8b-2c2b-44a1-a68c-7d9d1c0cdaa0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.688486 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abdd8b-2c2b-44a1-a68c-7d9d1c0cdaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.689003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abdd8b-2c2b-44a1-a68c-7d9d1c0cdaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.714258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef16275-2d89-45c7-9484-c1dde78e8039 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:03.771794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-35abdd8b-2c2b-44a1-a68c-7d9d1c0cdaa0 None None] MFA Rules not processed for user `f7196aa1e6ee4809a8a2a15d62cc7053`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:03.773937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-35abdd8b-2c2b-44a1-a68c-7d9d1c0cdaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:03.846744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4190757-c54e-4c82-b02a-e4ce364359c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:03.847511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190757-c54e-4c82-b02a-e4ce364359c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.847803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190757-c54e-4c82-b02a-e4ce364359c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.848090 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4190757-c54e-4c82-b02a-e4ce364359c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.848849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4190757-c54e-4c82-b02a-e4ce364359c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:03.850550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4190757-c54e-4c82-b02a-e4ce364359c9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:03.878706 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 817/1635] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:03.886890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4190757-c54e-4c82-b02a-e4ce364359c9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:03.902970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 819/1636] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:03.902970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abdd8b-2c2b-44a1-a68c-7d9d1c0cdaa0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:03.914977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abdd8b-2c2b-44a1-a68c-7d9d1c0cdaa0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:03.921872 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5902e-795c-4e5a-86a2-87f21bb3fba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:03.922573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5902e-795c-4e5a-86a2-87f21bb3fba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.923545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5902e-795c-4e5a-86a2-87f21bb3fba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.923839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5902e-795c-4e5a-86a2-87f21bb3fba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.924866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5902e-795c-4e5a-86a2-87f21bb3fba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:03.927627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5902e-795c-4e5a-86a2-87f21bb3fba4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:03.957624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 820/1637] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:03.958963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5902e-795c-4e5a-86a2-87f21bb3fba4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:03.975721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5902e-795c-4e5a-86a2-87f21bb3fba4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:03.982546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147344bd-4f4f-4feb-9637-eae1cbf53a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:03.983754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147344bd-4f4f-4feb-9637-eae1cbf53a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:03.984466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147344bd-4f4f-4feb-9637-eae1cbf53a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:03.984755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147344bd-4f4f-4feb-9637-eae1cbf53a2f admin admin] PATH_INFO: `/v3/projects/605016267c164bda8174a225d8943f76/users/1755834ded10440495f0e7a790a45339/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:03.989043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147344bd-4f4f-4feb-9637-eae1cbf53a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=605016267c164bda8174a225d8943f76, user_id=1755834ded10440495f0e7a790a45339, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:03.992581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147344bd-4f4f-4feb-9637-eae1cbf53a2f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:03.994108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:04.007194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4190757-c54e-4c82-b02a-e4ce364359c9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:04.007408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 821/1638] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:03 2022] PUT /identity/v3/projects/605016267c164bda8174a225d8943f76/users/1755834ded10440495f0e7a790a45339/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:04.008439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147344bd-4f4f-4feb-9637-eae1cbf53a2f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:04.042462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:04.110581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aec5c8-1612-4931-93c7-c1292aabd1dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:04.112269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aec5c8-1612-4931-93c7-c1292aabd1dd service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:04.112557 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aec5c8-1612-4931-93c7-c1292aabd1dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:04.114095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aec5c8-1612-4931-93c7-c1292aabd1dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:04.118034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147344bd-4f4f-4feb-9637-eae1cbf53a2f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:04.125075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf1abd-4560-48fb-817f-575d4957f23e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:04.125808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf1abd-4560-48fb-817f-575d4957f23e service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:04.126142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf1abd-4560-48fb-817f-575d4957f23e service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:04.126420 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf1abd-4560-48fb-817f-575d4957f23e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:04.269537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf1abd-4560-48fb-817f-575d4957f23e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:04.271856 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf1abd-4560-48fb-817f-575d4957f23e service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:04.287651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 822/1639] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:04.288734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf1abd-4560-48fb-817f-575d4957f23e service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:04.402601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf1abd-4560-48fb-817f-575d4957f23e service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:04.414657 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d8d46-ea83-4de4-83df-aa0646b9d019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:04.415711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d8d46-ea83-4de4-83df-aa0646b9d019 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:04.415991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d8d46-ea83-4de4-83df-aa0646b9d019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:04.416261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d8d46-ea83-4de4-83df-aa0646b9d019 admin admin] PATH_INFO: `/v3/projects/99b4b8d0934f4c2e9c9d5b92d144eb09/users/3433365739164957b4a911a699e32479/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:04.419486 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d8d46-ea83-4de4-83df-aa0646b9d019 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99b4b8d0934f4c2e9c9d5b92d144eb09, user_id=3433365739164957b4a911a699e32479, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:04.422435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d8d46-ea83-4de4-83df-aa0646b9d019 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:04.443301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aec5c8-1612-4931-93c7-c1292aabd1dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:04.445037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aec5c8-1612-4931-93c7-c1292aabd1dd service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:04.455655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 823/1640] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:04 2022] PUT /identity/v3/projects/99b4b8d0934f4c2e9c9d5b92d144eb09/users/3433365739164957b4a911a699e32479/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:04.455655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ec652-1c8f-4b8e-92f2-10072dcf727e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:04.455655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ec652-1c8f-4b8e-92f2-10072dcf727e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:04.455655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ec652-1c8f-4b8e-92f2-10072dcf727e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:04.545551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 818/1641] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:04.545551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28695de-e337-4de0-b487-91b1d51e482a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:04.545551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28695de-e337-4de0-b487-91b1d51e482a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:04.545551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28695de-e337-4de0-b487-91b1d51e482a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:04.549053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-df8ec652-1c8f-4b8e-92f2-10072dcf727e None None] MFA Rules not processed for user `1755834ded10440495f0e7a790a45339`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:04.549053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-df8ec652-1c8f-4b8e-92f2-10072dcf727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:04.574456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b28695de-e337-4de0-b487-91b1d51e482a None None] MFA Rules not processed for user `3433365739164957b4a911a699e32479`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:04.579188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b28695de-e337-4de0-b487-91b1d51e482a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:04.641478 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:04.718588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 824/1642] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:04.719157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ec652-1c8f-4b8e-92f2-10072dcf727e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:04.759228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 819/1643] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:04 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:04.862495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ec652-1c8f-4b8e-92f2-10072dcf727e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:04.868112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f5193-9cf6-4f95-a774-0de41303e0b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:04.871828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f5193-9cf6-4f95-a774-0de41303e0b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:04.872118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f5193-9cf6-4f95-a774-0de41303e0b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:04.872394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f5193-9cf6-4f95-a774-0de41303e0b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:05.023267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f5193-9cf6-4f95-a774-0de41303e0b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:05.024910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f5193-9cf6-4f95-a774-0de41303e0b6 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:05.043980 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 825/1644] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:05.495992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28695de-e337-4de0-b487-91b1d51e482a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:05.507521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28695de-e337-4de0-b487-91b1d51e482a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:05.517989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ecf86-d920-4c1f-9bb7-eac77405944b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:05.520817 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ecf86-d920-4c1f-9bb7-eac77405944b service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:05.521256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ecf86-d920-4c1f-9bb7-eac77405944b service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:05.521802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ecf86-d920-4c1f-9bb7-eac77405944b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:05.562284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ecf86-d920-4c1f-9bb7-eac77405944b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:05.564758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f5193-9cf6-4f95-a774-0de41303e0b6 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:05.577134 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ecf86-d920-4c1f-9bb7-eac77405944b service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:05.580318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f5193-9cf6-4f95-a774-0de41303e0b6 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:05.590361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb9f5c-5acd-44dd-8299-ccd0fb0276d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:05.592199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb9f5c-5acd-44dd-8299-ccd0fb0276d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:05.593895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb9f5c-5acd-44dd-8299-ccd0fb0276d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:05.595062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb9f5c-5acd-44dd-8299-ccd0fb0276d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:05.596462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 820/1645] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:05.632096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb9f5c-5acd-44dd-8299-ccd0fb0276d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:05.634939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb9f5c-5acd-44dd-8299-ccd0fb0276d5 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:05.662994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 826/1646] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:06.281139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ecf86-d920-4c1f-9bb7-eac77405944b service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:06.292949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ecf86-d920-4c1f-9bb7-eac77405944b service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:06.298112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5424262-6cb2-4ca5-8aab-1012d5ea95ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:06.299471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5424262-6cb2-4ca5-8aab-1012d5ea95ad service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:06.299933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5424262-6cb2-4ca5-8aab-1012d5ea95ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:06.300521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5424262-6cb2-4ca5-8aab-1012d5ea95ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:06.321055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:06.399936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5424262-6cb2-4ca5-8aab-1012d5ea95ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:06.401727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5424262-6cb2-4ca5-8aab-1012d5ea95ad service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:06.426388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 821/1647] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:06.713094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc81c4-30ca-470e-becd-7f2889108f3d None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:06.713094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc81c4-30ca-470e-becd-7f2889108f3d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:06.713094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc81c4-30ca-470e-becd-7f2889108f3d None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:06.713094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 827/1648] 10.222.0.27 () {54 vars in 961 bytes} [Wed Jun 29 08:35:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:35:06.723885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90131bbc-7661-4e61-bddc-a8c40c0e63d9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:06.724534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90131bbc-7661-4e61-bddc-a8c40c0e63d9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:06.724534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90131bbc-7661-4e61-bddc-a8c40c0e63d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:06.765521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-90131bbc-7661-4e61-bddc-a8c40c0e63d9 None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:06.767721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-90131bbc-7661-4e61-bddc-a8c40c0e63d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:06.965009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 822/1649] 10.222.0.27 () {58 vars in 1058 bytes} [Wed Jun 29 08:35:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:06.971796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263eecb-7917-4eb7-b0fc-058a3bd55efe None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:06.972364 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263eecb-7917-4eb7-b0fc-058a3bd55efe None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:06.972798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263eecb-7917-4eb7-b0fc-058a3bd55efe None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:06.974627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 828/1650] 10.222.0.27 () {54 vars in 912 bytes} [Wed Jun 29 08:35:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:35:06.981601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd3d6c-2b6d-4bde-9da4-c040af776f6c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:06.982052 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd3d6c-2b6d-4bde-9da4-c040af776f6c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:06.982482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd3d6c-2b6d-4bde-9da4-c040af776f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:07.038252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c0dd3d6c-2b6d-4bde-9da4-c040af776f6c None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:07.040812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c0dd3d6c-2b6d-4bde-9da4-c040af776f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:07.067768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 823/1651] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:35:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:07.094055 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263eecb-7917-4eb7-b0fc-058a3bd55efe None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:07.112123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263eecb-7917-4eb7-b0fc-058a3bd55efe None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:07.123689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b2bb8-ac25-4742-8859-383aac1d4c7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:07.124418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b2bb8-ac25-4742-8859-383aac1d4c7d service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:07.124713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b2bb8-ac25-4742-8859-383aac1d4c7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:07.124889 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b2bb8-ac25-4742-8859-383aac1d4c7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:07.147182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b2bb8-ac25-4742-8859-383aac1d4c7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:07.149405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b2bb8-ac25-4742-8859-383aac1d4c7d service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:07.181636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 829/1652] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:07.415646 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd3d6c-2b6d-4bde-9da4-c040af776f6c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:07.566468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd3d6c-2b6d-4bde-9da4-c040af776f6c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:07.581241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:07.591721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:07.592434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:07.593062 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] PATH_INFO: `/v3/users/75171062556f41e5af6b5b1c4d5da7b6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:07.595295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75171062556f41e5af6b5b1c4d5da7b6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:07.598076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: WARNING py.warnings [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:35:07.598076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: warnings.warn(msg) Jun 29 08:35:07.598076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: Jun 29 08:35:07.601137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:07.704581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:07.725179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75171062556f41e5af6b5b1c4d5da7b6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:07.726071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75171062556f41e5af6b5b1c4d5da7b6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:07.773388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75171062556f41e5af6b5b1c4d5da7b6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:07.780902 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75171062556f41e5af6b5b1c4d5da7b6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:07.817057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75171062556f41e5af6b5b1c4d5da7b6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:07.855642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 824/1653] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:35:07 2022] DELETE /identity/v3/users/75171062556f41e5af6b5b1c4d5da7b6 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:07.861192 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae306f-0c71-4854-b74b-cd93e2c8c960 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:07.861730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae306f-0c71-4854-b74b-cd93e2c8c960 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:07.861730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae306f-0c71-4854-b74b-cd93e2c8c960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:07.916259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-76ae306f-0c71-4854-b74b-cd93e2c8c960 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:07.919875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-76ae306f-0c71-4854-b74b-cd93e2c8c960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:08.169835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 830/1654] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:08.198683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:08.200214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:08.355318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b850a6-cb3f-4855-8a66-fbb731dc4d23 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:08.375375 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151af46-9a96-4e41-82ad-716ac91996cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:08.376175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151af46-9a96-4e41-82ad-716ac91996cd service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:08.376372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151af46-9a96-4e41-82ad-716ac91996cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:08.376652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151af46-9a96-4e41-82ad-716ac91996cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:08.394763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151af46-9a96-4e41-82ad-716ac91996cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:08.396271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151af46-9a96-4e41-82ad-716ac91996cd service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:08.417929 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 825/1655] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:10.046254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae306f-0c71-4854-b74b-cd93e2c8c960 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:10.048414 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-76ae306f-0c71-4854-b74b-cd93e2c8c960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:10.085138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae306f-0c71-4854-b74b-cd93e2c8c960 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:10.103705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:10.104643 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:10.106810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:10.107147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] PATH_INFO: `/v3/users/370e9262843b4ba7a907886fd6e08ac5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:10.111798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=370e9262843b4ba7a907886fd6e08ac5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:10.117025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:10.137381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:10.249861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370e9262843b4ba7a907886fd6e08ac5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:10.250553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370e9262843b4ba7a907886fd6e08ac5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:10.293632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370e9262843b4ba7a907886fd6e08ac5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:10.300710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370e9262843b4ba7a907886fd6e08ac5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:10.324800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370e9262843b4ba7a907886fd6e08ac5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:10.355202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 831/1656] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:35:10 2022] DELETE /identity/v3/users/370e9262843b4ba7a907886fd6e08ac5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:10.359540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51d0a5-4b2d-4d37-87d0-a3a2ca436cd5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:10.360371 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51d0a5-4b2d-4d37-87d0-a3a2ca436cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:10.360793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee51d0a5-4b2d-4d37-87d0-a3a2ca436cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:10.398163 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ee51d0a5-4b2d-4d37-87d0-a3a2ca436cd5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:10.400925 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ee51d0a5-4b2d-4d37-87d0-a3a2ca436cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:10.590276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 826/1657] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:10.605685 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:10.606672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51d0a5-4b2d-4d37-87d0-a3a2ca436cd5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:10.608398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:10.610269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ee51d0a5-4b2d-4d37-87d0-a3a2ca436cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:10.640350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee51d0a5-4b2d-4d37-87d0-a3a2ca436cd5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:10.663036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:10.665217 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:10.665506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:10.666199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] PATH_INFO: `/v3/projects/d7d568938e284206a571e256a12cac66` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:10.669959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7d568938e284206a571e256a12cac66)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:10.685091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:10.793958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20c5ce-c643-4b04-ba9a-b04b7b1e60c5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:10.803457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62d1e4-1a7d-4372-8753-0996e1495b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:10.804146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62d1e4-1a7d-4372-8753-0996e1495b0a service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:10.804428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62d1e4-1a7d-4372-8753-0996e1495b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:10.804701 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62d1e4-1a7d-4372-8753-0996e1495b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:10.840909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62d1e4-1a7d-4372-8753-0996e1495b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:10.842389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62d1e4-1a7d-4372-8753-0996e1495b0a service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:10.871330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7d568938e284206a571e256a12cac66', 'request_id': 'req-79b2810f-b5a3-4964-bdef-439c4ecb0e14'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:10.878736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 832/1658] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:10.901289 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7d568938e284206a571e256a12cac66', 'request_id': 'req-79b2810f-b5a3-4964-bdef-439c4ecb0e14'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:10.908580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:10.925170 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7d568938e284206a571e256a12cac66', 'request_id': 'req-79b2810f-b5a3-4964-bdef-439c4ecb0e14'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:10.968639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] The token cache is being invalidate because project d7d568938e284206a571e256a12cac66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:35:10.969043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:10.970885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:10.974000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 827/1659] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:35:10 2022] DELETE /identity/v3/projects/d7d568938e284206a571e256a12cac66 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:10.978408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62d1e4-1a7d-4372-8753-0996e1495b0a service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:10.995429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62d1e4-1a7d-4372-8753-0996e1495b0a service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:11.122946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:11.126117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:11.126117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:11.126117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] PATH_INFO: `/v3/users/1755834ded10440495f0e7a790a45339` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:11.129716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1755834ded10440495f0e7a790a45339)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:11.131765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:11.226412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:11.272669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1755834ded10440495f0e7a790a45339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:11.273887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1755834ded10440495f0e7a790a45339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:11.305754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1755834ded10440495f0e7a790a45339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:11.308082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1755834ded10440495f0e7a790a45339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:11.333029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1755834ded10440495f0e7a790a45339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:11.373118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 833/1660] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:35:10 2022] DELETE /identity/v3/users/1755834ded10440495f0e7a790a45339 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:13.037315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:13.039445 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:13.068492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2810f-b5a3-4964-bdef-439c4ecb0e14 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:13.193379 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:13.195074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:13.292304 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:13.293704 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:13.293989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:13.294271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] PATH_INFO: `/v3/projects/605016267c164bda8174a225d8943f76` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:13.302186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=605016267c164bda8174a225d8943f76)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:13.313718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:13.363634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddee3ae-bde1-4e27-83b3-6e1b6c340261 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:13.377397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:13.380218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:13.380218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:13.380435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] PATH_INFO: `/v3/projects/4e1c22413fe7410ab520a8891c1e0f65` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:13.384357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e1c22413fe7410ab520a8891c1e0f65)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:13.385292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: WARNING py.warnings [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 08:35:13.385292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: warnings.warn(msg) Jun 29 08:35:13.385292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: Jun 29 08:35:13.392753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:13.447830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '605016267c164bda8174a225d8943f76', 'request_id': 'req-fa5c1abd-64fb-42dc-85cc-265342fd61ba'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:13.461635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '605016267c164bda8174a225d8943f76', 'request_id': 'req-fa5c1abd-64fb-42dc-85cc-265342fd61ba'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:13.477232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '605016267c164bda8174a225d8943f76', 'request_id': 'req-fa5c1abd-64fb-42dc-85cc-265342fd61ba'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:13.510349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] The token cache is being invalidate because project 605016267c164bda8174a225d8943f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:35:13.510662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:13.511561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fa5c1abd-64fb-42dc-85cc-265342fd61ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:13.515005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 828/1661] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:35:13 2022] DELETE /identity/v3/projects/605016267c164bda8174a225d8943f76 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:13.520819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bb392-9750-49f0-a003-12049c6ea0fe None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:13.521111 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bb392-9750-49f0-a003-12049c6ea0fe None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:13.521383 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bb392-9750-49f0-a003-12049c6ea0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:13.550169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e1c22413fe7410ab520a8891c1e0f65', 'request_id': 'req-bf80bef5-a430-4368-ae31-61aafa716400'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:13.565124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-419bb392-9750-49f0-a003-12049c6ea0fe None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:13.566549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-419bb392-9750-49f0-a003-12049c6ea0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:13.589013 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e1c22413fe7410ab520a8891c1e0f65', 'request_id': 'req-bf80bef5-a430-4368-ae31-61aafa716400'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:13.605215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e1c22413fe7410ab520a8891c1e0f65', 'request_id': 'req-bf80bef5-a430-4368-ae31-61aafa716400'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:13.642753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] The token cache is being invalidate because project 4e1c22413fe7410ab520a8891c1e0f65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:35:13.643232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:13.644502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bf80bef5-a430-4368-ae31-61aafa716400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:13.647126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 834/1662] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:35:13 2022] DELETE /identity/v3/projects/4e1c22413fe7410ab520a8891c1e0f65 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:13.655786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d148a8-f0ea-46bc-be31-e05b046431ca None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:13.656253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d148a8-f0ea-46bc-be31-e05b046431ca None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:13.656586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d148a8-f0ea-46bc-be31-e05b046431ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:13.690734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b0d148a8-f0ea-46bc-be31-e05b046431ca None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:13.692057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b0d148a8-f0ea-46bc-be31-e05b046431ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:13.729549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 829/1663] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:35:13.739680 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:13.740103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e234fb-8151-40f0-92f9-77d3501b1bce None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:13.740433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e234fb-8151-40f0-92f9-77d3501b1bce None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:13.740673 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e234fb-8151-40f0-92f9-77d3501b1bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:13.790592 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-01e234fb-8151-40f0-92f9-77d3501b1bce None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:13.791819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-01e234fb-8151-40f0-92f9-77d3501b1bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:13.827456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 830/1664] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:13.831961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e234fb-8151-40f0-92f9-77d3501b1bce None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:13.838675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 835/1665] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:13.840582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c762bd-421c-4ba0-8b9d-308d759c6cc1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:13.840858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c762bd-421c-4ba0-8b9d-308d759c6cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:13.841141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c762bd-421c-4ba0-8b9d-308d759c6cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:13.854341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e234fb-8151-40f0-92f9-77d3501b1bce None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:13.874124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1db0b-f8a3-44e1-977f-8db38dae03c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:13.874858 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1db0b-f8a3-44e1-977f-8db38dae03c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:13.875150 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1db0b-f8a3-44e1-977f-8db38dae03c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:13.875430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1db0b-f8a3-44e1-977f-8db38dae03c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:13.876180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1db0b-f8a3-44e1-977f-8db38dae03c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:13.876493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1db0b-f8a3-44e1-977f-8db38dae03c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:13.878483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1db0b-f8a3-44e1-977f-8db38dae03c9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:13.884503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:13.885946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a9c762bd-421c-4ba0-8b9d-308d759c6cc1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:13.887349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a9c762bd-421c-4ba0-8b9d-308d759c6cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:13.914941 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 831/1666] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:35:13.918720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1db0b-f8a3-44e1-977f-8db38dae03c9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:13.919501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 836/1667] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:13.920477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c762bd-421c-4ba0-8b9d-308d759c6cc1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:13.945040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c762bd-421c-4ba0-8b9d-308d759c6cc1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:13.946460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1db0b-f8a3-44e1-977f-8db38dae03c9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:13.959636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ca64a-2d69-474a-95f1-76d211adb408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:13.960352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ca64a-2d69-474a-95f1-76d211adb408 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:13.962661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ca64a-2d69-474a-95f1-76d211adb408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:13.962661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ca64a-2d69-474a-95f1-76d211adb408 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:13.967488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c80e-4f76-4256-bfc0-cdf9818f2311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:13.968189 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c80e-4f76-4256-bfc0-cdf9818f2311 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:13.968472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c80e-4f76-4256-bfc0-cdf9818f2311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:13.968745 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c80e-4f76-4256-bfc0-cdf9818f2311 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:13.969453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2c80e-4f76-4256-bfc0-cdf9818f2311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:13.970897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ca64a-2d69-474a-95f1-76d211adb408 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:13.971466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2c80e-4f76-4256-bfc0-cdf9818f2311 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:13.973728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ca64a-2d69-474a-95f1-76d211adb408 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:13.976263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ca64a-2d69-474a-95f1-76d211adb408 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:14.020337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 837/1668] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:14.021531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ca64a-2d69-474a-95f1-76d211adb408 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:14.038895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ca64a-2d69-474a-95f1-76d211adb408 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:14.051555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136a837-581c-4057-8230-feca6ee8f70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:14.052536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136a837-581c-4057-8230-feca6ee8f70b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:14.052958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136a837-581c-4057-8230-feca6ee8f70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:14.053367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136a837-581c-4057-8230-feca6ee8f70b admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:14.054248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136a837-581c-4057-8230-feca6ee8f70b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:14.058140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 832/1669] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:13 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:14.062031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c80e-4f76-4256-bfc0-cdf9818f2311 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:14.063525 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136a837-581c-4057-8230-feca6ee8f70b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:14.137461 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 838/1670] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:14 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:14.138386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136a837-581c-4057-8230-feca6ee8f70b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:14.165835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c80e-4f76-4256-bfc0-cdf9818f2311 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:14.283129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136a837-581c-4057-8230-feca6ee8f70b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:14.298207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2efa4-40be-405c-b0ab-67acbfa96920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:14.298948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2efa4-40be-405c-b0ab-67acbfa96920 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:14.300215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2efa4-40be-405c-b0ab-67acbfa96920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:14.300215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2efa4-40be-405c-b0ab-67acbfa96920 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:14.300215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2efa4-40be-405c-b0ab-67acbfa96920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:14.302362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2efa4-40be-405c-b0ab-67acbfa96920 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:14.318987 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94297699-2a33-4376-ae73-d6fe9e8a49a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:14.320143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94297699-2a33-4376-ae73-d6fe9e8a49a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:14.320434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94297699-2a33-4376-ae73-d6fe9e8a49a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:14.320714 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94297699-2a33-4376-ae73-d6fe9e8a49a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:14.321820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94297699-2a33-4376-ae73-d6fe9e8a49a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:14.324312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94297699-2a33-4376-ae73-d6fe9e8a49a9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:14.341077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 839/1671] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:14 2022] POST /identity/v3/users => generated 282 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:14.348689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2efa4-40be-405c-b0ab-67acbfa96920 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:14.369732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 833/1672] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:14 2022] POST /identity/v3/users => generated 289 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:14.370651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94297699-2a33-4376-ae73-d6fe9e8a49a9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:14.370947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2efa4-40be-405c-b0ab-67acbfa96920 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:14.380009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67387640-6005-4b76-933d-1a98471ac604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:14.382096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67387640-6005-4b76-933d-1a98471ac604 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:14.383192 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67387640-6005-4b76-933d-1a98471ac604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:14.384245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67387640-6005-4b76-933d-1a98471ac604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:14.385331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67387640-6005-4b76-933d-1a98471ac604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:14.388714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67387640-6005-4b76-933d-1a98471ac604 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:14.390820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94297699-2a33-4376-ae73-d6fe9e8a49a9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:14.402540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b720013-79d1-4928-83ac-f2099788df25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:14.403262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b720013-79d1-4928-83ac-f2099788df25 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:14.403559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b720013-79d1-4928-83ac-f2099788df25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:14.403965 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b720013-79d1-4928-83ac-f2099788df25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:14.404471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 840/1673] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:14.407570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b720013-79d1-4928-83ac-f2099788df25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:14.408774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67387640-6005-4b76-933d-1a98471ac604 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:14.409214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b720013-79d1-4928-83ac-f2099788df25 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:14.420779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 834/1674] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:14.424904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b720013-79d1-4928-83ac-f2099788df25 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:14.436091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67387640-6005-4b76-933d-1a98471ac604 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:14.444185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b720013-79d1-4928-83ac-f2099788df25 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:14.451085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd47acb-2e9f-4847-bef4-69f227f2de08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:14.451266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452542d-79b2-435c-8ff9-55f942adb051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:14.451353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47acb-2e9f-4847-bef4-69f227f2de08 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:14.451587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47acb-2e9f-4847-bef4-69f227f2de08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:14.451675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452542d-79b2-435c-8ff9-55f942adb051 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:14.451845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452542d-79b2-435c-8ff9-55f942adb051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:14.452092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452542d-79b2-435c-8ff9-55f942adb051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:14.453325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452542d-79b2-435c-8ff9-55f942adb051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:14.454266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd47acb-2e9f-4847-bef4-69f227f2de08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:14.455025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452542d-79b2-435c-8ff9-55f942adb051 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:14.455145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd47acb-2e9f-4847-bef4-69f227f2de08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:14.460266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd47acb-2e9f-4847-bef4-69f227f2de08 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:14.474967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 835/1675] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:14.476172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452542d-79b2-435c-8ff9-55f942adb051 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:14.491292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 841/1676] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:14.492217 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd47acb-2e9f-4847-bef4-69f227f2de08 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:14.495186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452542d-79b2-435c-8ff9-55f942adb051 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:14.506684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c8142d-482d-4684-a383-01359f02ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:14.507434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8142d-482d-4684-a383-01359f02ad63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:14.509283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8142d-482d-4684-a383-01359f02ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:14.509283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c8142d-482d-4684-a383-01359f02ad63 admin admin] PATH_INFO: `/v3/projects/940ac7c4616941e2890d2e6943407027/users/e1abb107409a4854ab9d47ff176d84a5/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:14.515416 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd47acb-2e9f-4847-bef4-69f227f2de08 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:14.520314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31717a0d-1d4c-4653-9655-ed60c3a12950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:14.521230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31717a0d-1d4c-4653-9655-ed60c3a12950 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:14.521511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31717a0d-1d4c-4653-9655-ed60c3a12950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:14.521786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31717a0d-1d4c-4653-9655-ed60c3a12950 admin admin] PATH_INFO: `/v3/projects/3501580332b94c21bbe26de6965d4ee3/users/828ffde211084855b7be4268fa5fab32/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:14.548197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31717a0d-1d4c-4653-9655-ed60c3a12950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3501580332b94c21bbe26de6965d4ee3, user_id=828ffde211084855b7be4268fa5fab32, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:14.561042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8142d-482d-4684-a383-01359f02ad63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=940ac7c4616941e2890d2e6943407027, user_id=e1abb107409a4854ab9d47ff176d84a5, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:14.561042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c8142d-482d-4684-a383-01359f02ad63 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:14.577074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31717a0d-1d4c-4653-9655-ed60c3a12950 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:14.610630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 836/1677] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:14 2022] PUT /identity/v3/projects/940ac7c4616941e2890d2e6943407027/users/e1abb107409a4854ab9d47ff176d84a5/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:14.623714 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b60bb-0658-4e72-a072-501bb9d1f1ee None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:14.625065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b60bb-0658-4e72-a072-501bb9d1f1ee None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:14.625581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b60bb-0658-4e72-a072-501bb9d1f1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:14.647978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 842/1678] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:14 2022] PUT /identity/v3/projects/3501580332b94c21bbe26de6965d4ee3/users/828ffde211084855b7be4268fa5fab32/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:14.658752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163efdf-12e1-469c-955c-57dde6e01468 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:14.660721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163efdf-12e1-469c-955c-57dde6e01468 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:14.661184 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7163efdf-12e1-469c-955c-57dde6e01468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:14.721058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-423b60bb-0658-4e72-a072-501bb9d1f1ee None None] MFA Rules not processed for user `e1abb107409a4854ab9d47ff176d84a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:14.724806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-423b60bb-0658-4e72-a072-501bb9d1f1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:14.750384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7163efdf-12e1-469c-955c-57dde6e01468 None None] MFA Rules not processed for user `828ffde211084855b7be4268fa5fab32`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:14.752342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7163efdf-12e1-469c-955c-57dde6e01468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:14.776484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:14.845238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:14.932360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 837/1679] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:14.937179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b60bb-0658-4e72-a072-501bb9d1f1ee None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:14.955707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 843/1680] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:14 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:14.956374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163efdf-12e1-469c-955c-57dde6e01468 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:15.058577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b60bb-0658-4e72-a072-501bb9d1f1ee None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:15.073226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c3a85-4674-4cc5-8110-e4ac9f0482b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:15.079680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3a85-4674-4cc5-8110-e4ac9f0482b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:15.081106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3a85-4674-4cc5-8110-e4ac9f0482b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:15.083536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3a85-4674-4cc5-8110-e4ac9f0482b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:15.084452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c3a85-4674-4cc5-8110-e4ac9f0482b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:15.086554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c3a85-4674-4cc5-8110-e4ac9f0482b5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:15.091514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7163efdf-12e1-469c-955c-57dde6e01468 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:15.106812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0789a33-93e0-498f-b884-358b0d01e103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:15.107520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789a33-93e0-498f-b884-358b0d01e103 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:15.108217 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789a33-93e0-498f-b884-358b0d01e103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:15.108217 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0789a33-93e0-498f-b884-358b0d01e103 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:15.108848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0789a33-93e0-498f-b884-358b0d01e103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:15.111484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0789a33-93e0-498f-b884-358b0d01e103 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:15.155885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 844/1681] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:14 2022] POST /identity/v3/projects => generated 365 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:15.160949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0789a33-93e0-498f-b884-358b0d01e103 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:15.186091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 838/1682] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:14 2022] POST /identity/v3/projects => generated 375 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:15.191380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c3a85-4674-4cc5-8110-e4ac9f0482b5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:15.362803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0789a33-93e0-498f-b884-358b0d01e103 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:15.377309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c3a85-4674-4cc5-8110-e4ac9f0482b5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:15.377522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb0250-532a-4490-b4d6-ffe36206a08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:15.378773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb0250-532a-4490-b4d6-ffe36206a08c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:15.379064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb0250-532a-4490-b4d6-ffe36206a08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:15.379532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb0250-532a-4490-b4d6-ffe36206a08c admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:15.380384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb0250-532a-4490-b4d6-ffe36206a08c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:15.387595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb0250-532a-4490-b4d6-ffe36206a08c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:15.401303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e1137-d09c-4136-be7a-8651dd6c0125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:15.402536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e1137-d09c-4136-be7a-8651dd6c0125 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:15.402931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e1137-d09c-4136-be7a-8651dd6c0125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:15.403277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e1137-d09c-4136-be7a-8651dd6c0125 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:15.404696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e1137-d09c-4136-be7a-8651dd6c0125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:15.407627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e1137-d09c-4136-be7a-8651dd6c0125 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:15.454564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 845/1683] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:15 2022] POST /identity/v3/users => generated 289 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:15.456047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c0dc-04ed-4257-b302-ffd5432b802f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:15.456412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c0dc-04ed-4257-b302-ffd5432b802f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:15.456687 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c0dc-04ed-4257-b302-ffd5432b802f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:15.483374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 839/1684] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:15 2022] POST /identity/v3/users => generated 294 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:15.489130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc370e6-a1c4-4e96-9951-761dcab61428 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:15.489130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc370e6-a1c4-4e96-9951-761dcab61428 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:15.489130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc370e6-a1c4-4e96-9951-761dcab61428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:15.523759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-30c2c0dc-04ed-4257-b302-ffd5432b802f None None] MFA Rules not processed for user `d55c898180bd4cbf910f3d92045e7d67`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:15.525071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-30c2c0dc-04ed-4257-b302-ffd5432b802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:15.550953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1dc370e6-a1c4-4e96-9951-761dcab61428 None None] MFA Rules not processed for user `bccf002d9da34d1dad0ee71b5b7bb1c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:15.553904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1dc370e6-a1c4-4e96-9951-761dcab61428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:15.579287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:15.612350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:15.657572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 840/1685] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:15.672453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc370e6-a1c4-4e96-9951-761dcab61428 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:15.713697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 846/1686] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:15.716511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2c0dc-04ed-4257-b302-ffd5432b802f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:15.726851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc370e6-a1c4-4e96-9951-761dcab61428 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:15.733979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c3afb-c5f1-4c15-8b5f-375bc29d3858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:15.737305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c3afb-c5f1-4c15-8b5f-375bc29d3858 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:15.737819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c3afb-c5f1-4c15-8b5f-375bc29d3858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:15.738895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c3afb-c5f1-4c15-8b5f-375bc29d3858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:15.742780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c3afb-c5f1-4c15-8b5f-375bc29d3858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:15.745064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c3afb-c5f1-4c15-8b5f-375bc29d3858 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:15.747286 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2c0dc-04ed-4257-b302-ffd5432b802f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:15.761245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1e83f-4872-4c06-92bc-d2df5db17cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:15.761952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1e83f-4872-4c06-92bc-d2df5db17cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:15.764067 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1e83f-4872-4c06-92bc-d2df5db17cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:15.764366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1e83f-4872-4c06-92bc-d2df5db17cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:15.767262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1e83f-4872-4c06-92bc-d2df5db17cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:15.771637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 841/1687] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:15.775365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1e83f-4872-4c06-92bc-d2df5db17cc2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:15.779845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c3afb-c5f1-4c15-8b5f-375bc29d3858 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:15.788028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-181c3afb-c5f1-4c15-8b5f-375bc29d3858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:15.791441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 847/1688] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:15.792745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b1600-553c-4310-bf2d-5d405dd40df0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:15.793035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b1600-553c-4310-bf2d-5d405dd40df0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:15.793312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b1600-553c-4310-bf2d-5d405dd40df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:15.829924 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f30b1600-553c-4310-bf2d-5d405dd40df0 None None] MFA Rules not processed for user `d55c898180bd4cbf910f3d92045e7d67`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:15.832818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f30b1600-553c-4310-bf2d-5d405dd40df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:15.870183 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 848/1689] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:15.874556 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b1600-553c-4310-bf2d-5d405dd40df0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:15.899143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b1600-553c-4310-bf2d-5d405dd40df0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:15.914625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc27677-b929-4128-ad0b-f3946299930c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:15.915566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc27677-b929-4128-ad0b-f3946299930c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:15.915846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc27677-b929-4128-ad0b-f3946299930c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:15.916117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc27677-b929-4128-ad0b-f3946299930c admin admin] PATH_INFO: `/v3/projects/76c989ae0bba45b487e7c4a147af9eb8/users/a897b644867d48239ad2cdc1a7107a9d/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:15.963753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc27677-b929-4128-ad0b-f3946299930c admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c989ae0bba45b487e7c4a147af9eb8, user_id=a897b644867d48239ad2cdc1a7107a9d, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:15.966794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc27677-b929-4128-ad0b-f3946299930c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:16.004503 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 849/1690] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:15 2022] PUT /identity/v3/projects/76c989ae0bba45b487e7c4a147af9eb8/users/a897b644867d48239ad2cdc1a7107a9d/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:16.004503 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc27677-b929-4128-ad0b-f3946299930c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:16.006646 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c3afb-c5f1-4c15-8b5f-375bc29d3858 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:16.121424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc27677-b929-4128-ad0b-f3946299930c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:16.131630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522dc5e-c427-46de-9f91-b17070cca8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:16.133334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522dc5e-c427-46de-9f91-b17070cca8e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:16.134204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef59a89-0029-44eb-af11-92825ca4c84c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:16.135059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef59a89-0029-44eb-af11-92825ca4c84c service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:16.135487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522dc5e-c427-46de-9f91-b17070cca8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:16.135858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4522dc5e-c427-46de-9f91-b17070cca8e1 admin admin] PATH_INFO: `/v3/projects/5e12b1a75cd8466b86d317386c0ea980/users/e712bd9b2497416ba1e7b37b74b7c073/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:16.136280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef59a89-0029-44eb-af11-92825ca4c84c service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:16.137322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef59a89-0029-44eb-af11-92825ca4c84c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:16.167682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4522dc5e-c427-46de-9f91-b17070cca8e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e12b1a75cd8466b86d317386c0ea980, user_id=e712bd9b2497416ba1e7b37b74b7c073, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:16.172005 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4522dc5e-c427-46de-9f91-b17070cca8e1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:16.209090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 850/1691] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:16 2022] PUT /identity/v3/projects/5e12b1a75cd8466b86d317386c0ea980/users/e712bd9b2497416ba1e7b37b74b7c073/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:16.209090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522dc5e-c427-46de-9f91-b17070cca8e1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:16.211382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:16.211503 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4522dc5e-c427-46de-9f91-b17070cca8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:16.249522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef59a89-0029-44eb-af11-92825ca4c84c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:16.257760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef59a89-0029-44eb-af11-92825ca4c84c service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:16.281937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 842/1692] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:16.282841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef59a89-0029-44eb-af11-92825ca4c84c service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:16.309333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef59a89-0029-44eb-af11-92825ca4c84c service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:16.458825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9e46-b331-47bd-8f91-e28073610a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:16.463393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9e46-b331-47bd-8f91-e28073610a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:16.463821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9e46-b331-47bd-8f91-e28073610a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:16.467447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9e46-b331-47bd-8f91-e28073610a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:16.472621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be9e46-b331-47bd-8f91-e28073610a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:16.482612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be9e46-b331-47bd-8f91-e28073610a41 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:16.493173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4522dc5e-c427-46de-9f91-b17070cca8e1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:16.519796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75dc813-dae7-48b6-ab92-396c496f079e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:16.520335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dc813-dae7-48b6-ab92-396c496f079e service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:16.520617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dc813-dae7-48b6-ab92-396c496f079e service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:16.520888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75dc813-dae7-48b6-ab92-396c496f079e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:16.543832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 843/1693] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:16.544850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9e46-b331-47bd-8f91-e28073610a41 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:16.564318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:16.587652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9e46-b331-47bd-8f91-e28073610a41 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:16.598773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebae86-da31-4570-b6ec-8e77e6a37f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:16.602779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebae86-da31-4570-b6ec-8e77e6a37f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:16.603189 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebae86-da31-4570-b6ec-8e77e6a37f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:16.603638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebae86-da31-4570-b6ec-8e77e6a37f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:16.608982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebae86-da31-4570-b6ec-8e77e6a37f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:16.610911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebae86-da31-4570-b6ec-8e77e6a37f00 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:16.661399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 844/1694] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:16.662470 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebae86-da31-4570-b6ec-8e77e6a37f00 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:16.694331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebae86-da31-4570-b6ec-8e77e6a37f00 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:16.723622 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52869bc-5d23-4342-b902-cf45562c5602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:16.724549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52869bc-5d23-4342-b902-cf45562c5602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:16.724833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52869bc-5d23-4342-b902-cf45562c5602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:16.729122 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52869bc-5d23-4342-b902-cf45562c5602 admin admin] PATH_INFO: `/v3/projects/76c989ae0bba45b487e7c4a147af9eb8/users/a897b644867d48239ad2cdc1a7107a9d/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:16.739208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52869bc-5d23-4342-b902-cf45562c5602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c989ae0bba45b487e7c4a147af9eb8, user_id=a897b644867d48239ad2cdc1a7107a9d, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:16.751238 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52869bc-5d23-4342-b902-cf45562c5602 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:16.768007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75dc813-dae7-48b6-ab92-396c496f079e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:16.770527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75dc813-dae7-48b6-ab92-396c496f079e service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:16.809469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 845/1695] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:16 2022] PUT /identity/v3/projects/76c989ae0bba45b487e7c4a147af9eb8/users/a897b644867d48239ad2cdc1a7107a9d/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:16.809469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52869bc-5d23-4342-b902-cf45562c5602 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:16.809798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 851/1696] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:16.813251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb0b7f-3de5-45f2-a64a-c9a5781f4359 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:16.813512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb0b7f-3de5-45f2-a64a-c9a5781f4359 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:16.815157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb0b7f-3de5-45f2-a64a-c9a5781f4359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:16.942942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52869bc-5d23-4342-b902-cf45562c5602 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:16.955297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f6eb0b7f-3de5-45f2-a64a-c9a5781f4359 None None] MFA Rules not processed for user `a897b644867d48239ad2cdc1a7107a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:16.960544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f6eb0b7f-3de5-45f2-a64a-c9a5781f4359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:16.962645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39748c-68b3-4d55-be7b-5ce2a3c150a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:16.966424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39748c-68b3-4d55-be7b-5ce2a3c150a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:16.970406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39748c-68b3-4d55-be7b-5ce2a3c150a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:16.970832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39748c-68b3-4d55-be7b-5ce2a3c150a3 admin admin] PATH_INFO: `/v3/projects/5e12b1a75cd8466b86d317386c0ea980/users/e712bd9b2497416ba1e7b37b74b7c073/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:16.979845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39748c-68b3-4d55-be7b-5ce2a3c150a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e12b1a75cd8466b86d317386c0ea980, user_id=e712bd9b2497416ba1e7b37b74b7c073, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:16.996331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39748c-68b3-4d55-be7b-5ce2a3c150a3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:17.058361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 846/1697] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:16 2022] PUT /identity/v3/projects/5e12b1a75cd8466b86d317386c0ea980/users/e712bd9b2497416ba1e7b37b74b7c073/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:17.064396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39748c-68b3-4d55-be7b-5ce2a3c150a3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:17.074214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7d39748c-68b3-4d55-be7b-5ce2a3c150a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:17.232636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:17.238119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 852/1698] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:17.243203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc857de4-43d6-45dd-97df-dc069f198ceb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:17.243203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc857de4-43d6-45dd-97df-dc069f198ceb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:17.243557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc857de4-43d6-45dd-97df-dc069f198ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:17.312605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39748c-68b3-4d55-be7b-5ce2a3c150a3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:17.322260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4ef7e-ebcd-48b1-a0fe-e4867d1569c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:17.326614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4ef7e-ebcd-48b1-a0fe-e4867d1569c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:17.327029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4ef7e-ebcd-48b1-a0fe-e4867d1569c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:17.327379 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4ef7e-ebcd-48b1-a0fe-e4867d1569c3 admin admin] PATH_INFO: `/v3/users/a1c9e67668b74cb49a124da506ee03b5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:17.329992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4ef7e-ebcd-48b1-a0fe-e4867d1569c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1c9e67668b74cb49a124da506ee03b5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:17.335784 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4ef7e-ebcd-48b1-a0fe-e4867d1569c3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:17.337602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fc857de4-43d6-45dd-97df-dc069f198ceb None None] MFA Rules not processed for user `e712bd9b2497416ba1e7b37b74b7c073`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:17.338946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fc857de4-43d6-45dd-97df-dc069f198ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:17.495263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-18e4ef7e-ebcd-48b1-a0fe-e4867d1569c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1c9e67668b74cb49a124da506ee03b5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:17.496158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-18e4ef7e-ebcd-48b1-a0fe-e4867d1569c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1c9e67668b74cb49a124da506ee03b5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:17.502053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 853/1699] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:17.502053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc857de4-43d6-45dd-97df-dc069f198ceb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:17.507526 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fc857de4-43d6-45dd-97df-dc069f198ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:17.538099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-18e4ef7e-ebcd-48b1-a0fe-e4867d1569c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1c9e67668b74cb49a124da506ee03b5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:17.543304 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-18e4ef7e-ebcd-48b1-a0fe-e4867d1569c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1c9e67668b74cb49a124da506ee03b5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:17.551823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:17.570104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-18e4ef7e-ebcd-48b1-a0fe-e4867d1569c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1c9e67668b74cb49a124da506ee03b5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:17.621681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 847/1700] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:35:17 2022] DELETE /identity/v3/users/a1c9e67668b74cb49a124da506ee03b5 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:17.626588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ce8ce-1ee3-4e5c-a822-4df9affaf936 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:17.627150 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ce8ce-1ee3-4e5c-a822-4df9affaf936 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:17.627597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ce8ce-1ee3-4e5c-a822-4df9affaf936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:17.637040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc857de4-43d6-45dd-97df-dc069f198ceb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:17.710931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-224ce8ce-1ee3-4e5c-a822-4df9affaf936 None None] MFA Rules not processed for user `ac984d1f6efa403e85483bd7fa78000e`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:17.712324 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-224ce8ce-1ee3-4e5c-a822-4df9affaf936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:17.754233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06091a51-c4f0-4fb4-b3a1-73325d6c225a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:17.755046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091a51-c4f0-4fb4-b3a1-73325d6c225a service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:17.755398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091a51-c4f0-4fb4-b3a1-73325d6c225a service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:17.755740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06091a51-c4f0-4fb4-b3a1-73325d6c225a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:17.766449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-06091a51-c4f0-4fb4-b3a1-73325d6c225a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:17.924330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06091a51-c4f0-4fb4-b3a1-73325d6c225a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:17.926804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06091a51-c4f0-4fb4-b3a1-73325d6c225a service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:17.943623 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 854/1701] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:17.944998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abae846-86ca-4b60-864f-5eb21f1ec805 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:17.945265 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abae846-86ca-4b60-864f-5eb21f1ec805 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:17.945545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abae846-86ca-4b60-864f-5eb21f1ec805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:18.000893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6abae846-86ca-4b60-864f-5eb21f1ec805 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:18.004887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6abae846-86ca-4b60-864f-5eb21f1ec805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:18.009761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 848/1702] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:18.011480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406f34c-b45f-49a4-be2e-3f68904b116b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:18.023302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406f34c-b45f-49a4-be2e-3f68904b116b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:18.023302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406f34c-b45f-49a4-be2e-3f68904b116b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:18.103421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3406f34c-b45f-49a4-be2e-3f68904b116b None None] MFA Rules not processed for user `ac984d1f6efa403e85483bd7fa78000e`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:18.105610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3406f34c-b45f-49a4-be2e-3f68904b116b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:18.139046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 849/1703] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:18.140110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406f34c-b45f-49a4-be2e-3f68904b116b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:18.143452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3406f34c-b45f-49a4-be2e-3f68904b116b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:18.174024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406f34c-b45f-49a4-be2e-3f68904b116b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:18.187324 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604a025-16c6-4595-81ea-f419c8a69bde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:18.188440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604a025-16c6-4595-81ea-f419c8a69bde service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:18.189019 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604a025-16c6-4595-81ea-f419c8a69bde service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:18.189481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604a025-16c6-4595-81ea-f419c8a69bde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:18.194036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c604a025-16c6-4595-81ea-f419c8a69bde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:18.212116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 855/1704] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:18.213325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abae846-86ca-4b60-864f-5eb21f1ec805 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:18.218056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6abae846-86ca-4b60-864f-5eb21f1ec805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:18.301910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:18.306850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:18.345655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604a025-16c6-4595-81ea-f419c8a69bde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:18.352142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604a025-16c6-4595-81ea-f419c8a69bde service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:18.386725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 850/1705] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:18.387946 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604a025-16c6-4595-81ea-f419c8a69bde service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:18.390157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c604a025-16c6-4595-81ea-f419c8a69bde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:18.438868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abae846-86ca-4b60-864f-5eb21f1ec805 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:18.456645 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a90f24-9b78-471c-9124-f6ea8fa31c18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:18.457414 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a90f24-9b78-471c-9124-f6ea8fa31c18 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:18.458549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a90f24-9b78-471c-9124-f6ea8fa31c18 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:18.458984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a90f24-9b78-471c-9124-f6ea8fa31c18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:18.514758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a90f24-9b78-471c-9124-f6ea8fa31c18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:18.517718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a90f24-9b78-471c-9124-f6ea8fa31c18 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:18.566854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 856/1706] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:18.618637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604a025-16c6-4595-81ea-f419c8a69bde service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:18.627675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055ec50-f74e-40dd-bbf4-7f34b548735a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:18.628590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055ec50-f74e-40dd-bbf4-7f34b548735a service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:18.629071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055ec50-f74e-40dd-bbf4-7f34b548735a service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:18.629358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055ec50-f74e-40dd-bbf4-7f34b548735a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:18.666510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055ec50-f74e-40dd-bbf4-7f34b548735a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:18.668282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f055ec50-f74e-40dd-bbf4-7f34b548735a service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:18.687290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 851/1707] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:18.716929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41974ec6-14ae-4776-930e-bbf21aec8b55 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:18.716929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41974ec6-14ae-4776-930e-bbf21aec8b55 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:18.717116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41974ec6-14ae-4776-930e-bbf21aec8b55 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:18.718678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 857/1708] 10.222.0.27 () {54 vars in 912 bytes} [Wed Jun 29 08:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:35:18.724726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e35ba-8754-46e0-ae4f-149e38b06450 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:18.725294 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e35ba-8754-46e0-ae4f-149e38b06450 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:18.725809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e35ba-8754-46e0-ae4f-149e38b06450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:18.893017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0c0e35ba-8754-46e0-ae4f-149e38b06450 None None] MFA Rules not processed for user `49a1439029fa4b719bc114c3f59e0050`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:18.899641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0c0e35ba-8754-46e0-ae4f-149e38b06450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:18.925153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 852/1709] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:35:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:18.959706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41974ec6-14ae-4776-930e-bbf21aec8b55 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:18.962176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-41974ec6-14ae-4776-930e-bbf21aec8b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:19.045014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41974ec6-14ae-4776-930e-bbf21aec8b55 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:19.062369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5049386-2702-40ab-9d29-ed67c90df265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:19.074848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5049386-2702-40ab-9d29-ed67c90df265 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:19.075779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5049386-2702-40ab-9d29-ed67c90df265 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:19.076417 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5049386-2702-40ab-9d29-ed67c90df265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:19.103713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5049386-2702-40ab-9d29-ed67c90df265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:19.107917 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5049386-2702-40ab-9d29-ed67c90df265 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:19.137600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 858/1710] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:20.022977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e35ba-8754-46e0-ae4f-149e38b06450 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:20.024427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0c0e35ba-8754-46e0-ae4f-149e38b06450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:20.059931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e35ba-8754-46e0-ae4f-149e38b06450 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:20.075946 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:20.076875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:20.077158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:20.077436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] PATH_INFO: `/v3/users/828ffde211084855b7be4268fa5fab32` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:20.080205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=828ffde211084855b7be4268fa5fab32)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:20.083059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:20.105710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd240b85-d9d9-45d9-ba9f-f952bad9339c None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:20.106199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd240b85-d9d9-45d9-ba9f-f952bad9339c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:20.106269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd240b85-d9d9-45d9-ba9f-f952bad9339c None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:20.107979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 859/1711] 10.222.0.27 () {54 vars in 912 bytes} [Wed Jun 29 08:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:35:20.115717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea9f38-9142-449d-bcf0-47ad602c75d3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:20.116351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea9f38-9142-449d-bcf0-47ad602c75d3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:20.117521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea9f38-9142-449d-bcf0-47ad602c75d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:20.166097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6dea9f38-9142-449d-bcf0-47ad602c75d3 None None] MFA Rules not processed for user `49a1439029fa4b719bc114c3f59e0050`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:20.169094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6dea9f38-9142-449d-bcf0-47ad602c75d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:20.180019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 860/1712] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:35:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:20.213824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea9f38-9142-449d-bcf0-47ad602c75d3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:20.222089 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '828ffde211084855b7be4268fa5fab32'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:20.223524 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '828ffde211084855b7be4268fa5fab32'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:20.241848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea9f38-9142-449d-bcf0-47ad602c75d3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:20.248155 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54f841-3606-4c80-98c2-983f173b63e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:20.250861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54f841-3606-4c80-98c2-983f173b63e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:20.251649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '828ffde211084855b7be4268fa5fab32'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:20.252501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54f841-3606-4c80-98c2-983f173b63e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:20.252989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54f841-3606-4c80-98c2-983f173b63e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:20.254993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '828ffde211084855b7be4268fa5fab32'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:20.256076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cf54f841-3606-4c80-98c2-983f173b63e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:20.280041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '828ffde211084855b7be4268fa5fab32'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:20.287134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54f841-3606-4c80-98c2-983f173b63e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:20.292696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54f841-3606-4c80-98c2-983f173b63e3 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:20.321150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 861/1713] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:35:20.321531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 853/1714] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:35:20 2022] DELETE /identity/v3/users/828ffde211084855b7be4268fa5fab32 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:20.328567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f62224-25f2-4405-a98a-296504cdba50 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:20.329002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f62224-25f2-4405-a98a-296504cdba50 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:20.329437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f62224-25f2-4405-a98a-296504cdba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:20.370802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f3f62224-25f2-4405-a98a-296504cdba50 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:20.373401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f3f62224-25f2-4405-a98a-296504cdba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:20.507281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 862/1715] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:20.567629 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:20.569188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:20.696853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:20.707735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f62224-25f2-4405-a98a-296504cdba50 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:20.709478 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f3f62224-25f2-4405-a98a-296504cdba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:20.731166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f62224-25f2-4405-a98a-296504cdba50 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:20.736739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8316f-d81d-4fb2-ab28-5046d80d45a1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:20.742568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:20.743521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:20.743774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:20.744047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] PATH_INFO: `/v3/projects/38f7be8ca176417c8edd2b5f778c08c5` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:20.745314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e845d1eb-a0e1-43da-8463-9d9564f1c3ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:20.746139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845d1eb-a0e1-43da-8463-9d9564f1c3ad service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:20.746441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845d1eb-a0e1-43da-8463-9d9564f1c3ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:20.746738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e845d1eb-a0e1-43da-8463-9d9564f1c3ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:20.748172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38f7be8ca176417c8edd2b5f778c08c5)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:20.750151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:20.767550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845d1eb-a0e1-43da-8463-9d9564f1c3ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:20.771546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e845d1eb-a0e1-43da-8463-9d9564f1c3ad service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:20.795908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 854/1716] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:20.817751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efafb31-4c67-4ddf-baef-6de73907d13f None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:20.818047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efafb31-4c67-4ddf-baef-6de73907d13f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:20.818333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efafb31-4c67-4ddf-baef-6de73907d13f None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:20.820101 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 855/1717] 10.222.0.27 () {54 vars in 960 bytes} [Wed Jun 29 08:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 08:35:20.832571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0a32a-2ba0-4074-9113-d06a27ea4ecd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:20.832861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0a32a-2ba0-4074-9113-d06a27ea4ecd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:20.833146 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0a32a-2ba0-4074-9113-d06a27ea4ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:20.875589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38f7be8ca176417c8edd2b5f778c08c5', 'request_id': 'req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:20.892229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-04f0a32a-2ba0-4074-9113-d06a27ea4ecd None None] MFA Rules not processed for user `00a08f31638d4eaab5ca83d6f493db6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:20.892229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-04f0a32a-2ba0-4074-9113-d06a27ea4ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:20.892568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38f7be8ca176417c8edd2b5f778c08c5', 'request_id': 'req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:20.906172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38f7be8ca176417c8edd2b5f778c08c5', 'request_id': 'req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:20.936208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] The token cache is being invalidate because project 38f7be8ca176417c8edd2b5f778c08c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:35:20.937385 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:20.944215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-89a512cb-e67b-4fc4-9ea6-c2046a9dc257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:20.957441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 863/1718] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:35:20 2022] DELETE /identity/v3/projects/38f7be8ca176417c8edd2b5f778c08c5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:20.965159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55ec8b-3418-4be1-99d4-7c75e764b531 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:20.965495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55ec8b-3418-4be1-99d4-7c75e764b531 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:20.965737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55ec8b-3418-4be1-99d4-7c75e764b531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:20.974230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:21.022923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2a55ec8b-3418-4be1-99d4-7c75e764b531 None None] MFA Rules not processed for user `37d6c5d1f76445499566472213f13e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:21.024426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2a55ec8b-3418-4be1-99d4-7c75e764b531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:21.054391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 856/1719] 10.222.0.27 () {58 vars in 1057 bytes} [Wed Jun 29 08:35:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:21.056317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0a32a-2ba0-4074-9113-d06a27ea4ecd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:21.061906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-04f0a32a-2ba0-4074-9113-d06a27ea4ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:21.114227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 864/1720] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:21.115799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55ec8b-3418-4be1-99d4-7c75e764b531 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:21.137513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55ec8b-3418-4be1-99d4-7c75e764b531 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:21.144576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf9a08-700f-469c-9eaf-d6b48771b26d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:21.146968 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf9a08-700f-469c-9eaf-d6b48771b26d service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:21.147607 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf9a08-700f-469c-9eaf-d6b48771b26d service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:21.148103 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cf9a08-700f-469c-9eaf-d6b48771b26d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:21.151260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-08cf9a08-700f-469c-9eaf-d6b48771b26d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:21.170570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf9a08-700f-469c-9eaf-d6b48771b26d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:21.172928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cf9a08-700f-469c-9eaf-d6b48771b26d service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:21.179846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0a32a-2ba0-4074-9113-d06a27ea4ecd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:21.186251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0033f-15a3-4604-a6f3-218c687924a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:21.187525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0033f-15a3-4604-a6f3-218c687924a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:21.187881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0033f-15a3-4604-a6f3-218c687924a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:21.188218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0033f-15a3-4604-a6f3-218c687924a5 admin admin] PATH_INFO: `/v3/users/3433365739164957b4a911a699e32479` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:21.189787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0033f-15a3-4604-a6f3-218c687924a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3433365739164957b4a911a699e32479)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:21.192258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0033f-15a3-4604-a6f3-218c687924a5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:21.196410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 865/1721] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:21.198489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf9a08-700f-469c-9eaf-d6b48771b26d service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:21.201058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-08cf9a08-700f-469c-9eaf-d6b48771b26d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:21.281230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbc0033f-15a3-4604-a6f3-218c687924a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3433365739164957b4a911a699e32479'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:21.282876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbc0033f-15a3-4604-a6f3-218c687924a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3433365739164957b4a911a699e32479'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:21.301369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbc0033f-15a3-4604-a6f3-218c687924a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3433365739164957b4a911a699e32479'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:21.303447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbc0033f-15a3-4604-a6f3-218c687924a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3433365739164957b4a911a699e32479'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:21.319401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cbc0033f-15a3-4604-a6f3-218c687924a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3433365739164957b4a911a699e32479'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:21.389424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 857/1722] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:35:21 2022] DELETE /identity/v3/users/3433365739164957b4a911a699e32479 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:21.404859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cf9a08-700f-469c-9eaf-d6b48771b26d service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:21.631525 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696ec6a-2c7b-49b3-8d96-f8f4b36e851d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:21.632748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696ec6a-2c7b-49b3-8d96-f8f4b36e851d service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:21.633130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696ec6a-2c7b-49b3-8d96-f8f4b36e851d service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:21.633502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696ec6a-2c7b-49b3-8d96-f8f4b36e851d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:21.636419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6696ec6a-2c7b-49b3-8d96-f8f4b36e851d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:21.657982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:21.762052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696ec6a-2c7b-49b3-8d96-f8f4b36e851d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:21.764984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696ec6a-2c7b-49b3-8d96-f8f4b36e851d service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:21.796665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 866/1723] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:21.919670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466b0e4-943e-4b5e-8e71-36ff54e043fc None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:21.920652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466b0e4-943e-4b5e-8e71-36ff54e043fc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:21.921352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466b0e4-943e-4b5e-8e71-36ff54e043fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:21.952449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:22.003662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d466b0e4-943e-4b5e-8e71-36ff54e043fc None None] MFA Rules not processed for user `ac984d1f6efa403e85483bd7fa78000e`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:22.006522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d466b0e4-943e-4b5e-8e71-36ff54e043fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:22.091553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696ec6a-2c7b-49b3-8d96-f8f4b36e851d service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:22.100018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6696ec6a-2c7b-49b3-8d96-f8f4b36e851d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:22.325855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 858/1724] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:22.379203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696ec6a-2c7b-49b3-8d96-f8f4b36e851d service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:22.393448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:22.394467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:22.394775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:22.395206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] PATH_INFO: `/v3/projects/3501580332b94c21bbe26de6965d4ee3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:22.399501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3501580332b94c21bbe26de6965d4ee3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:22.401468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:22.531880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3501580332b94c21bbe26de6965d4ee3', 'request_id': 'req-f1b56a13-ea36-4496-87d3-6f28ae9bd533'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:22.581753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3501580332b94c21bbe26de6965d4ee3', 'request_id': 'req-f1b56a13-ea36-4496-87d3-6f28ae9bd533'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:22.594242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3501580332b94c21bbe26de6965d4ee3', 'request_id': 'req-f1b56a13-ea36-4496-87d3-6f28ae9bd533'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:22.648414 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] The token cache is being invalidate because project 3501580332b94c21bbe26de6965d4ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:35:22.650178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:22.653984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:22.658007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 867/1725] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:35:22 2022] DELETE /identity/v3/projects/3501580332b94c21bbe26de6965d4ee3 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:35:22.664961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466b0e4-943e-4b5e-8e71-36ff54e043fc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:22.691429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466b0e4-943e-4b5e-8e71-36ff54e043fc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:22.899852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d5885-ac48-4b14-9a79-3b2b6b428f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:22.902287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d5885-ac48-4b14-9a79-3b2b6b428f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:22.904125 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d5885-ac48-4b14-9a79-3b2b6b428f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:22.904234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d5885-ac48-4b14-9a79-3b2b6b428f2e admin admin] PATH_INFO: `/v3/users/a897b644867d48239ad2cdc1a7107a9d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:22.908415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d5885-ac48-4b14-9a79-3b2b6b428f2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a897b644867d48239ad2cdc1a7107a9d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:22.919245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d5885-ac48-4b14-9a79-3b2b6b428f2e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:23.112569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cb5d5885-ac48-4b14-9a79-3b2b6b428f2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a897b644867d48239ad2cdc1a7107a9d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:23.113962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cb5d5885-ac48-4b14-9a79-3b2b6b428f2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a897b644867d48239ad2cdc1a7107a9d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:23.193665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cb5d5885-ac48-4b14-9a79-3b2b6b428f2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a897b644867d48239ad2cdc1a7107a9d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:23.194550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cb5d5885-ac48-4b14-9a79-3b2b6b428f2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a897b644867d48239ad2cdc1a7107a9d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:23.231849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-cb5d5885-ac48-4b14-9a79-3b2b6b428f2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a897b644867d48239ad2cdc1a7107a9d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:23.288261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 859/1726] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:35:22 2022] DELETE /identity/v3/users/a897b644867d48239ad2cdc1a7107a9d => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:23.548426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:23.549992 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:23.584147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56a13-ea36-4496-87d3-6f28ae9bd533 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:23.716570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:23.781344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:23.782406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:23.783892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:23.784260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] PATH_INFO: `/v3/projects/99b4b8d0934f4c2e9c9d5b92d144eb09` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:23.789243 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99b4b8d0934f4c2e9c9d5b92d144eb09)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:23.791272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:23.945259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99b4b8d0934f4c2e9c9d5b92d144eb09', 'request_id': 'req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:23.954885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99b4b8d0934f4c2e9c9d5b92d144eb09', 'request_id': 'req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:23.968191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99b4b8d0934f4c2e9c9d5b92d144eb09', 'request_id': 'req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:24.010430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] The token cache is being invalidate because project 99b4b8d0934f4c2e9c9d5b92d144eb09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:35:24.010825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:24.011670 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-de58f4f5-e9ee-47d8-8bf8-0250917c73f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:24.021197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 868/1727] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:35:23 2022] DELETE /identity/v3/projects/99b4b8d0934f4c2e9c9d5b92d144eb09 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:24.023116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d56df-8a01-49e8-922a-951906cb54e9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:24.025426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d56df-8a01-49e8-922a-951906cb54e9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:24.025725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d56df-8a01-49e8-922a-951906cb54e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:24.058883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b12d56df-8a01-49e8-922a-951906cb54e9 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:24.060055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b12d56df-8a01-49e8-922a-951906cb54e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:24.111074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:24.164306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 860/1728] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:24.172945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb91541-bea3-46d5-a4f6-27332ba4c997 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:24.173716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb91541-bea3-46d5-a4f6-27332ba4c997 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:24.174129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb91541-bea3-46d5-a4f6-27332ba4c997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:24.202734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-deb91541-bea3-46d5-a4f6-27332ba4c997 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:24.204843 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-deb91541-bea3-46d5-a4f6-27332ba4c997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:24.217503 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 869/1729] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:35:24.221647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d56df-8a01-49e8-922a-951906cb54e9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:24.235544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d56df-8a01-49e8-922a-951906cb54e9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:24.241043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177b3dd-2d36-41eb-aea6-99b9696d4089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:24.241944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177b3dd-2d36-41eb-aea6-99b9696d4089 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:24.242395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177b3dd-2d36-41eb-aea6-99b9696d4089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:24.242827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177b3dd-2d36-41eb-aea6-99b9696d4089 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:24.243884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177b3dd-2d36-41eb-aea6-99b9696d4089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:24.244328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177b3dd-2d36-41eb-aea6-99b9696d4089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:24.247483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177b3dd-2d36-41eb-aea6-99b9696d4089 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:24.262342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 861/1730] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:24.267444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb91541-bea3-46d5-a4f6-27332ba4c997 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:24.280311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb91541-bea3-46d5-a4f6-27332ba4c997 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:24.290549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e13086-411d-423b-951b-fc7a8e872f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:24.292766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e13086-411d-423b-951b-fc7a8e872f28 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:24.293156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e13086-411d-423b-951b-fc7a8e872f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:24.293532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e13086-411d-423b-951b-fc7a8e872f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:24.294914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e13086-411d-423b-951b-fc7a8e872f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:24.298572 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e13086-411d-423b-951b-fc7a8e872f28 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:24.340037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 870/1731] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:24 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:24.345186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177b3dd-2d36-41eb-aea6-99b9696d4089 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:24.368558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177b3dd-2d36-41eb-aea6-99b9696d4089 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:24.522104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985ab48-18bf-4903-9ecc-490c7dd32591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:24.527120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985ab48-18bf-4903-9ecc-490c7dd32591 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:24.527573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985ab48-18bf-4903-9ecc-490c7dd32591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:24.527991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985ab48-18bf-4903-9ecc-490c7dd32591 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:24.528948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985ab48-18bf-4903-9ecc-490c7dd32591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:24.534284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985ab48-18bf-4903-9ecc-490c7dd32591 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:24.591349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 862/1732] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:24 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:24.592694 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e13086-411d-423b-951b-fc7a8e872f28 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:24.612387 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e13086-411d-423b-951b-fc7a8e872f28 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:24.626849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df7bd8-3d9a-4f27-83e1-f965c78ac61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:24.628736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df7bd8-3d9a-4f27-83e1-f965c78ac61b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:24.631329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df7bd8-3d9a-4f27-83e1-f965c78ac61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:24.632045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df7bd8-3d9a-4f27-83e1-f965c78ac61b admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:24.634161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df7bd8-3d9a-4f27-83e1-f965c78ac61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:24.636731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6df7bd8-3d9a-4f27-83e1-f965c78ac61b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:24.690403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 871/1733] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:24.696511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985ab48-18bf-4903-9ecc-490c7dd32591 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:24.725044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985ab48-18bf-4903-9ecc-490c7dd32591 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:24.750276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80b330-7b82-4b46-8cbb-377d608824e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:24.753915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80b330-7b82-4b46-8cbb-377d608824e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:24.754152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80b330-7b82-4b46-8cbb-377d608824e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:24.756400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80b330-7b82-4b46-8cbb-377d608824e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:24.760178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80b330-7b82-4b46-8cbb-377d608824e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:24.767769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80b330-7b82-4b46-8cbb-377d608824e6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:24.814978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 863/1734] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:24.816914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df7bd8-3d9a-4f27-83e1-f965c78ac61b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:24.843711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6df7bd8-3d9a-4f27-83e1-f965c78ac61b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:24.856928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80b330-7b82-4b46-8cbb-377d608824e6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:24.858897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6a80b330-7b82-4b46-8cbb-377d608824e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:24.861805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f912df-0ca4-4e62-81dd-7ba3e9c31bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:24.862882 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f912df-0ca4-4e62-81dd-7ba3e9c31bce admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:24.863605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f912df-0ca4-4e62-81dd-7ba3e9c31bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:24.863605 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f912df-0ca4-4e62-81dd-7ba3e9c31bce admin admin] PATH_INFO: `/v3/projects/7eda322434244f0cb317b36b99f5f6ec/users/6c137b92885547baba30e8005b3c89db/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:24.885932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80b330-7b82-4b46-8cbb-377d608824e6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:24.905340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:24.906448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:24.906747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:24.907030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] PATH_INFO: `/v3/projects/76c989ae0bba45b487e7c4a147af9eb8` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:24.909679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c989ae0bba45b487e7c4a147af9eb8)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:24.911866 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:24.939222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f912df-0ca4-4e62-81dd-7ba3e9c31bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eda322434244f0cb317b36b99f5f6ec, user_id=6c137b92885547baba30e8005b3c89db, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:24.942182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f912df-0ca4-4e62-81dd-7ba3e9c31bce admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:24.987545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 872/1735] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:24 2022] PUT /identity/v3/projects/7eda322434244f0cb317b36b99f5f6ec/users/6c137b92885547baba30e8005b3c89db/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:24.989142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625fd30-6cf0-4de3-bc1b-2cd4b6d4295b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:24.991120 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625fd30-6cf0-4de3-bc1b-2cd4b6d4295b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:24.991120 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625fd30-6cf0-4de3-bc1b-2cd4b6d4295b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:25.053495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4625fd30-6cf0-4de3-bc1b-2cd4b6d4295b None None] MFA Rules not processed for user `bccf002d9da34d1dad0ee71b5b7bb1c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:25.054740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4625fd30-6cf0-4de3-bc1b-2cd4b6d4295b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:25.095426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c989ae0bba45b487e7c4a147af9eb8', 'request_id': 'req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:25.145526 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c989ae0bba45b487e7c4a147af9eb8', 'request_id': 'req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:25.174559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c989ae0bba45b487e7c4a147af9eb8', 'request_id': 'req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:25.221231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] The token cache is being invalidate because project 76c989ae0bba45b487e7c4a147af9eb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:35:25.221770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:25.222584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b858e048-cf9a-4841-9fc7-e28e5f0b2c01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:25.227543 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 864/1736] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:35:24 2022] DELETE /identity/v3/projects/76c989ae0bba45b487e7c4a147af9eb8 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:35:25.230385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80783b61-7e0c-493c-8ceb-e28219722463 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:25.236359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80783b61-7e0c-493c-8ceb-e28219722463 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:25.239656 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80783b61-7e0c-493c-8ceb-e28219722463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:25.269147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 873/1737] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:25.285597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c383a7f-3afd-462b-8307-d184677253ce None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:25.287900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c383a7f-3afd-462b-8307-d184677253ce None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:25.288203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c383a7f-3afd-462b-8307-d184677253ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:25.359733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-80783b61-7e0c-493c-8ceb-e28219722463 None None] MFA Rules not processed for user `6c137b92885547baba30e8005b3c89db`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:25.360112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5c383a7f-3afd-462b-8307-d184677253ce None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:25.361822 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5c383a7f-3afd-462b-8307-d184677253ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:25.367282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-80783b61-7e0c-493c-8ceb-e28219722463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:25.382566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:25.391780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:25.526998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 874/1738] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:25.534750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829a664-a976-4183-8f20-4ebda365e406 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:25.535114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829a664-a976-4183-8f20-4ebda365e406 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:25.535457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829a664-a976-4183-8f20-4ebda365e406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:25.597306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7829a664-a976-4183-8f20-4ebda365e406 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:25.601061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7829a664-a976-4183-8f20-4ebda365e406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:25.633418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 875/1739] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:25.637622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829a664-a976-4183-8f20-4ebda365e406 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:25.657473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829a664-a976-4183-8f20-4ebda365e406 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:25.679197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7cb07-673f-4d4a-add1-00a550557f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:25.681383 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 865/1740] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:25.681520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7cb07-673f-4d4a-add1-00a550557f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:25.681807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7cb07-673f-4d4a-add1-00a550557f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:25.684802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80783b61-7e0c-493c-8ceb-e28219722463 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:25.685305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c7cb07-673f-4d4a-add1-00a550557f4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:25.688547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7cb07-673f-4d4a-add1-00a550557f4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:25.689168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7cb07-673f-4d4a-add1-00a550557f4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:25.691930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-80783b61-7e0c-493c-8ceb-e28219722463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:25.701937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c7cb07-673f-4d4a-add1-00a550557f4d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:25.729783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80783b61-7e0c-493c-8ceb-e28219722463 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:25.741043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cba435-e918-48b6-bb37-6c35919d6e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:25.741961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cba435-e918-48b6-bb37-6c35919d6e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:25.742417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cba435-e918-48b6-bb37-6c35919d6e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:25.742897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cba435-e918-48b6-bb37-6c35919d6e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:25.743841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cba435-e918-48b6-bb37-6c35919d6e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:25.747046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cba435-e918-48b6-bb37-6c35919d6e9a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:25.747384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 876/1741] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:25.748609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7cb07-673f-4d4a-add1-00a550557f4d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:25.798074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c7cb07-673f-4d4a-add1-00a550557f4d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:25.810201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 866/1742] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:25 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:25.814181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cba435-e918-48b6-bb37-6c35919d6e9a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:25.822327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3200fbb-7a63-4bf5-bf0c-ebd368b5d588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:25.823747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3200fbb-7a63-4bf5-bf0c-ebd368b5d588 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:25.824069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3200fbb-7a63-4bf5-bf0c-ebd368b5d588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:25.824350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3200fbb-7a63-4bf5-bf0c-ebd368b5d588 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:25.825112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3200fbb-7a63-4bf5-bf0c-ebd368b5d588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:25.834479 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3200fbb-7a63-4bf5-bf0c-ebd368b5d588 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:25.930679 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 877/1743] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:25 2022] POST /identity/v3/projects => generated 369 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:25.935446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3200fbb-7a63-4bf5-bf0c-ebd368b5d588 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:26.006977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cba435-e918-48b6-bb37-6c35919d6e9a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:26.143953 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3200fbb-7a63-4bf5-bf0c-ebd368b5d588 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:26.163824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e43d84-7f53-4dfc-812a-6624261c2e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:26.164166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e43d84-7f53-4dfc-812a-6624261c2e7c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:26.164452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e43d84-7f53-4dfc-812a-6624261c2e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:26.164729 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e43d84-7f53-4dfc-812a-6624261c2e7c admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:26.165511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e43d84-7f53-4dfc-812a-6624261c2e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:26.167424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e43d84-7f53-4dfc-812a-6624261c2e7c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:26.215866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 878/1744] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:25 2022] POST /identity/v3/users => generated 285 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:26.219756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e43d84-7f53-4dfc-812a-6624261c2e7c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:26.232118 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe309981-f65e-4c5e-9530-9bbdd8b6796a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:26.236354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe309981-f65e-4c5e-9530-9bbdd8b6796a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:26.236654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe309981-f65e-4c5e-9530-9bbdd8b6796a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:26.236936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe309981-f65e-4c5e-9530-9bbdd8b6796a admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:26.237802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe309981-f65e-4c5e-9530-9bbdd8b6796a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:26.254054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e43d84-7f53-4dfc-812a-6624261c2e7c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:26.272843 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60986f1-8dec-491f-8a13-8fdd7138c648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:26.273568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60986f1-8dec-491f-8a13-8fdd7138c648 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:26.273851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60986f1-8dec-491f-8a13-8fdd7138c648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:26.274145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60986f1-8dec-491f-8a13-8fdd7138c648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:26.274904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60986f1-8dec-491f-8a13-8fdd7138c648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:26.276553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60986f1-8dec-491f-8a13-8fdd7138c648 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:26.278439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe309981-f65e-4c5e-9530-9bbdd8b6796a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:26.300032 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 879/1745] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:35:26.311473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4bc00-8c93-4da7-97c2-a170c19ff4b2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:26.312058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4bc00-8c93-4da7-97c2-a170c19ff4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:26.312058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4bc00-8c93-4da7-97c2-a170c19ff4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:26.341357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-01d4bc00-8c93-4da7-97c2-a170c19ff4b2 None None] MFA Rules not processed for user `37d6c5d1f76445499566472213f13e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:26.343043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-01d4bc00-8c93-4da7-97c2-a170c19ff4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:26.348960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 867/1746] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:25 2022] POST /identity/v3/users => generated 298 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:26.348960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe309981-f65e-4c5e-9530-9bbdd8b6796a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:26.383733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe309981-f65e-4c5e-9530-9bbdd8b6796a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:26.411222 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a683113-b567-40b0-ae70-90ea6c08136d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:26.412924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a683113-b567-40b0-ae70-90ea6c08136d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:26.413218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a683113-b567-40b0-ae70-90ea6c08136d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:26.413501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a683113-b567-40b0-ae70-90ea6c08136d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:26.414255 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a683113-b567-40b0-ae70-90ea6c08136d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:26.420146 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a683113-b567-40b0-ae70-90ea6c08136d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:26.486588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 868/1747] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:26.488098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a683113-b567-40b0-ae70-90ea6c08136d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:26.522036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a683113-b567-40b0-ae70-90ea6c08136d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:26.550183 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:26.551341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf5840-0e3d-45f7-bb44-62e477edfa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:26.552531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf5840-0e3d-45f7-bb44-62e477edfa32 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:26.553503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf5840-0e3d-45f7-bb44-62e477edfa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:26.556848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf5840-0e3d-45f7-bb44-62e477edfa32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:26.576756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf5840-0e3d-45f7-bb44-62e477edfa32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:26.582358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf5840-0e3d-45f7-bb44-62e477edfa32 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:26.590483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:26.594219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 880/1748] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:26.594219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4bc00-8c93-4da7-97c2-a170c19ff4b2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:26.615386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 869/1749] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:26.617232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4bc00-8c93-4da7-97c2-a170c19ff4b2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:26.619067 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf5840-0e3d-45f7-bb44-62e477edfa32 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:26.636934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05315bb6-f17c-4069-b065-a5142620e07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:26.637930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05315bb6-f17c-4069-b065-a5142620e07d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:26.639347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05315bb6-f17c-4069-b065-a5142620e07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:26.639625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05315bb6-f17c-4069-b065-a5142620e07d admin admin] PATH_INFO: `/v3/projects/5b7cc5a5a53643d18369c280323a3bea/users/59e705d023ca4ea5844139b85506fcca/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:26.645085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf5840-0e3d-45f7-bb44-62e477edfa32 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:26.671184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aa8cb-7076-4da7-b943-7ee3a0a7d363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:26.674139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa8cb-7076-4da7-b943-7ee3a0a7d363 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:26.675523 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa8cb-7076-4da7-b943-7ee3a0a7d363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:26.677537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa8cb-7076-4da7-b943-7ee3a0a7d363 admin admin] PATH_INFO: `/v3/projects/25657cdd64a844dd869d948094851345/users/2256a3d1362f4b0891a74707e68e920c/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:26.692409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05315bb6-f17c-4069-b065-a5142620e07d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7cc5a5a53643d18369c280323a3bea, user_id=59e705d023ca4ea5844139b85506fcca, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:26.705307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05315bb6-f17c-4069-b065-a5142620e07d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:26.783365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 881/1750] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:26 2022] PUT /identity/v3/projects/5b7cc5a5a53643d18369c280323a3bea/users/59e705d023ca4ea5844139b85506fcca/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:26.797506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee805b6-f282-497c-9d56-d13975a0bd6d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:26.798168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee805b6-f282-497c-9d56-d13975a0bd6d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:26.798613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee805b6-f282-497c-9d56-d13975a0bd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:26.799178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9aa8cb-7076-4da7-b943-7ee3a0a7d363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25657cdd64a844dd869d948094851345, user_id=2256a3d1362f4b0891a74707e68e920c, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:26.812693 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9aa8cb-7076-4da7-b943-7ee3a0a7d363 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:26.838836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-aee805b6-f282-497c-9d56-d13975a0bd6d None None] MFA Rules not processed for user `59e705d023ca4ea5844139b85506fcca`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:26.841007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-aee805b6-f282-497c-9d56-d13975a0bd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:26.869248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 870/1751] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:26 2022] PUT /identity/v3/projects/25657cdd64a844dd869d948094851345/users/2256a3d1362f4b0891a74707e68e920c/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:26.873415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aa8cb-7076-4da7-b943-7ee3a0a7d363 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:27.050891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 882/1752] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:35:27.055620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee805b6-f282-497c-9d56-d13975a0bd6d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:27.140836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aa8cb-7076-4da7-b943-7ee3a0a7d363 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:27.142615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee805b6-f282-497c-9d56-d13975a0bd6d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:27.148225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7897c-2d5b-4eb8-a547-81e7b1cd1c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:27.150202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7897c-2d5b-4eb8-a547-81e7b1cd1c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:27.151777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7897c-2d5b-4eb8-a547-81e7b1cd1c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:27.152235 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b7897c-2d5b-4eb8-a547-81e7b1cd1c93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:27.154090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7897c-2d5b-4eb8-a547-81e7b1cd1c93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:27.162935 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b7897c-2d5b-4eb8-a547-81e7b1cd1c93 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:27.163630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e16e4-f1fa-4553-bdc0-e376a7cb3502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:27.164512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e16e4-f1fa-4553-bdc0-e376a7cb3502 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:27.164963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e16e4-f1fa-4553-bdc0-e376a7cb3502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:27.167132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e16e4-f1fa-4553-bdc0-e376a7cb3502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:27.168942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e16e4-f1fa-4553-bdc0-e376a7cb3502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:27.191977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e16e4-f1fa-4553-bdc0-e376a7cb3502 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:27.224306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 871/1753] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:27.229905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e16e4-f1fa-4553-bdc0-e376a7cb3502 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:27.240411 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 883/1754] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:27 2022] POST /identity/v3/projects => generated 369 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:27.241760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7897c-2d5b-4eb8-a547-81e7b1cd1c93 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:27.281326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e16e4-f1fa-4553-bdc0-e376a7cb3502 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:27.398168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0b123-1758-43fd-9415-2f449bcaf16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:27.404634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0b123-1758-43fd-9415-2f449bcaf16f admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:27.405626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0b123-1758-43fd-9415-2f449bcaf16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:27.405827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:27.407211 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0b123-1758-43fd-9415-2f449bcaf16f admin admin] PATH_INFO: `/v3/projects/25657cdd64a844dd869d948094851345/users/2256a3d1362f4b0891a74707e68e920c/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:27.412841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0b123-1758-43fd-9415-2f449bcaf16f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25657cdd64a844dd869d948094851345, user_id=2256a3d1362f4b0891a74707e68e920c, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:27.415964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0b123-1758-43fd-9415-2f449bcaf16f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:27.439521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b7897c-2d5b-4eb8-a547-81e7b1cd1c93 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:27.455102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 872/1755] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:27 2022] PUT /identity/v3/projects/25657cdd64a844dd869d948094851345/users/2256a3d1362f4b0891a74707e68e920c/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:27.463766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfdfe-91e7-4966-89e8-15cea6514af4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:27.464325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfdfe-91e7-4966-89e8-15cea6514af4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:27.465172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bfdfe-91e7-4966-89e8-15cea6514af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:27.590185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-389bfdfe-91e7-4966-89e8-15cea6514af4 None None] MFA Rules not processed for user `2256a3d1362f4b0891a74707e68e920c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:27.591503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-389bfdfe-91e7-4966-89e8-15cea6514af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:27.696208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe2e8b-a6b1-495a-a280-f46a774297c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:27.696753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe2e8b-a6b1-495a-a280-f46a774297c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:27.697026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe2e8b-a6b1-495a-a280-f46a774297c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:27.697484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe2e8b-a6b1-495a-a280-f46a774297c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:27.699149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe2e8b-a6b1-495a-a280-f46a774297c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:27.701137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe2e8b-a6b1-495a-a280-f46a774297c4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:27.717927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:27.762014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 884/1756] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:27 2022] POST /identity/v3/users => generated 291 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:27.767612 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe2e8b-a6b1-495a-a280-f46a774297c4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:27.773970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-07fe2e8b-a6b1-495a-a280-f46a774297c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:27.780884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 873/1757] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:27.782197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bfdfe-91e7-4966-89e8-15cea6514af4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:27.811110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bfdfe-91e7-4966-89e8-15cea6514af4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:27.824689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea6aef-5d2d-4fb0-a377-74720d2e4390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:27.825379 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea6aef-5d2d-4fb0-a377-74720d2e4390 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:27.825669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea6aef-5d2d-4fb0-a377-74720d2e4390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:27.825991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea6aef-5d2d-4fb0-a377-74720d2e4390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:27.826749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea6aef-5d2d-4fb0-a377-74720d2e4390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:27.828389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea6aef-5d2d-4fb0-a377-74720d2e4390 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:27.869804 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 874/1758] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:27.870892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea6aef-5d2d-4fb0-a377-74720d2e4390 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:27.878264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d6ea6aef-5d2d-4fb0-a377-74720d2e4390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:27.930748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe2e8b-a6b1-495a-a280-f46a774297c4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:27.939662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c17372-e4e6-43c9-a6bc-d7058fd2a593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:27.940997 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c17372-e4e6-43c9-a6bc-d7058fd2a593 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:27.942321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c17372-e4e6-43c9-a6bc-d7058fd2a593 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:27.942657 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c17372-e4e6-43c9-a6bc-d7058fd2a593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:27.945887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-15c17372-e4e6-43c9-a6bc-d7058fd2a593 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:27.968322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea6aef-5d2d-4fb0-a377-74720d2e4390 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:27.977140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fef4c5-818d-4c82-b136-f5f572f180ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:27.979424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fef4c5-818d-4c82-b136-f5f572f180ee service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:27.980749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fef4c5-818d-4c82-b136-f5f572f180ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:27.981061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fef4c5-818d-4c82-b136-f5f572f180ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:28.158249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c17372-e4e6-43c9-a6bc-d7058fd2a593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:28.161074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fef4c5-818d-4c82-b136-f5f572f180ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:28.167792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c17372-e4e6-43c9-a6bc-d7058fd2a593 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:28.167940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fef4c5-818d-4c82-b136-f5f572f180ee service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:28.195527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 875/1759] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:28.196224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fef4c5-818d-4c82-b136-f5f572f180ee service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:28.216211 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 885/1760] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:28.236388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fef4c5-818d-4c82-b136-f5f572f180ee service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:28.257807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8af488-48c7-45fe-8545-bef57b6221c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:28.258870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8af488-48c7-45fe-8545-bef57b6221c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:28.259178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8af488-48c7-45fe-8545-bef57b6221c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:28.264282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8af488-48c7-45fe-8545-bef57b6221c1 admin admin] PATH_INFO: `/v3/projects/c38027e126fe4e7cb669cd2bfc0eb290/users/fd6b02aff3724b0d8dd0cc85b50f5eaa/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:28.333666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8af488-48c7-45fe-8545-bef57b6221c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38027e126fe4e7cb669cd2bfc0eb290, user_id=fd6b02aff3724b0d8dd0cc85b50f5eaa, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:28.338034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8af488-48c7-45fe-8545-bef57b6221c1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:28.382602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 876/1761] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:28 2022] PUT /identity/v3/projects/c38027e126fe4e7cb669cd2bfc0eb290/users/fd6b02aff3724b0d8dd0cc85b50f5eaa/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:28.386944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c17372-e4e6-43c9-a6bc-d7058fd2a593 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:28.418442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c17372-e4e6-43c9-a6bc-d7058fd2a593 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:28.596453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:28.663182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bce9f3-2069-41a9-b493-c721e9173ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:28.666527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bce9f3-2069-41a9-b493-c721e9173ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:28.666872 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bce9f3-2069-41a9-b493-c721e9173ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:28.667206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bce9f3-2069-41a9-b493-c721e9173ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:28.676198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bce9f3-2069-41a9-b493-c721e9173ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:28.680276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bce9f3-2069-41a9-b493-c721e9173ae4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:28.711874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 886/1762] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:28.716314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8af488-48c7-45fe-8545-bef57b6221c1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:28.739290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8af488-48c7-45fe-8545-bef57b6221c1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:28.762936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f1dce-17c6-48a9-bd3a-c8a83cd1ea78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:28.763876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f1dce-17c6-48a9-bd3a-c8a83cd1ea78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:28.764161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f1dce-17c6-48a9-bd3a-c8a83cd1ea78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:28.770362 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f1dce-17c6-48a9-bd3a-c8a83cd1ea78 admin admin] PATH_INFO: `/v3/projects/c38027e126fe4e7cb669cd2bfc0eb290/users/fd6b02aff3724b0d8dd0cc85b50f5eaa/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:28.780883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f1dce-17c6-48a9-bd3a-c8a83cd1ea78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c38027e126fe4e7cb669cd2bfc0eb290, user_id=fd6b02aff3724b0d8dd0cc85b50f5eaa, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:28.786926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005f1dce-17c6-48a9-bd3a-c8a83cd1ea78 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:28.845299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 877/1763] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:28 2022] PUT /identity/v3/projects/c38027e126fe4e7cb669cd2bfc0eb290/users/fd6b02aff3724b0d8dd0cc85b50f5eaa/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:28.845758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ad9e6-b46b-4a2f-aee3-d69283777237 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:28.846053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ad9e6-b46b-4a2f-aee3-d69283777237 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:28.846353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ad9e6-b46b-4a2f-aee3-d69283777237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:28.915803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d07ad9e6-b46b-4a2f-aee3-d69283777237 None None] MFA Rules not processed for user `fd6b02aff3724b0d8dd0cc85b50f5eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:28.918584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d07ad9e6-b46b-4a2f-aee3-d69283777237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:28.974365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da34f8-2bde-4f77-bb2e-1a8cd3936e53 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:28.974672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da34f8-2bde-4f77-bb2e-1a8cd3936e53 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:28.975044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da34f8-2bde-4f77-bb2e-1a8cd3936e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:29.016080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-72da34f8-2bde-4f77-bb2e-1a8cd3936e53 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:29.021952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-72da34f8-2bde-4f77-bb2e-1a8cd3936e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:29.206151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 887/1764] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:28 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:29.281854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:29.310522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 878/1765] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:35:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:35:29.411692 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ad9e6-b46b-4a2f-aee3-d69283777237 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:29.614875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ad9e6-b46b-4a2f-aee3-d69283777237 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:29.630592 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee30576-58b6-4a10-9728-b8575bdbc293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:29.633929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee30576-58b6-4a10-9728-b8575bdbc293 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:29.635043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee30576-58b6-4a10-9728-b8575bdbc293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:29.636058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee30576-58b6-4a10-9728-b8575bdbc293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:29.647770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da34f8-2bde-4f77-bb2e-1a8cd3936e53 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:29.649477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-72da34f8-2bde-4f77-bb2e-1a8cd3936e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:29.677358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee30576-58b6-4a10-9728-b8575bdbc293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:29.679734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee30576-58b6-4a10-9728-b8575bdbc293 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:29.700556 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 888/1766] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:29 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:29.978812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da34f8-2bde-4f77-bb2e-1a8cd3936e53 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:29.995520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c92e69-fbe9-420e-a04c-7b501a925390 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:29.996327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c92e69-fbe9-420e-a04c-7b501a925390 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:29.996616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c92e69-fbe9-420e-a04c-7b501a925390 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:29.996994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c92e69-fbe9-420e-a04c-7b501a925390 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:30.024620 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c92e69-fbe9-420e-a04c-7b501a925390 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:30.026256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c92e69-fbe9-420e-a04c-7b501a925390 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:30.052025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 879/1767] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:35:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 29 08:35:33.320687 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee30576-58b6-4a10-9728-b8575bdbc293 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:33.408105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee30576-58b6-4a10-9728-b8575bdbc293 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:33.414284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:33.415596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:33.416032 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:33.416613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] PATH_INFO: `/v3/users/59e705d023ca4ea5844139b85506fcca` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:33.418051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] RBAC: Authorizing `identity:delete_user(user_id=59e705d023ca4ea5844139b85506fcca)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:33.420497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:33.422972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:33.515054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59e705d023ca4ea5844139b85506fcca'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:33.517634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59e705d023ca4ea5844139b85506fcca'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:33.543256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59e705d023ca4ea5844139b85506fcca'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:33.544067 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59e705d023ca4ea5844139b85506fcca'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:33.553532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59e705d023ca4ea5844139b85506fcca'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:33.575598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 889/1768] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:35:33 2022] DELETE /identity/v3/users/59e705d023ca4ea5844139b85506fcca => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:35:33.579986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e732-7898-45ac-8492-9d87653b0c9e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:33.580505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e732-7898-45ac-8492-9d87653b0c9e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:33.580595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e732-7898-45ac-8492-9d87653b0c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:33.607413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5551e732-7898-45ac-8492-9d87653b0c9e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:33.608644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5551e732-7898-45ac-8492-9d87653b0c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:33.671055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 880/1769] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:33.680830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:33.682241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:33.743524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97970bc3-0d16-402a-a33b-93a083eed3df admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:33.747065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ee645-9533-4e9c-a646-938f8f82b5b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:33.747710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee645-9533-4e9c-a646-938f8f82b5b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:33.747994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee645-9533-4e9c-a646-938f8f82b5b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:33.748265 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee645-9533-4e9c-a646-938f8f82b5b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:33.757993 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ee645-9533-4e9c-a646-938f8f82b5b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:33.759545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ee645-9533-4e9c-a646-938f8f82b5b0 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:33.769869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 890/1770] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:34.071047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551e732-7898-45ac-8492-9d87653b0c9e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:34.072547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5551e732-7898-45ac-8492-9d87653b0c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:34.093365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551e732-7898-45ac-8492-9d87653b0c9e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:34.098724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:34.099623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:34.099914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:34.100302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] PATH_INFO: `/v3/projects/5b7cc5a5a53643d18369c280323a3bea` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:34.101396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7cc5a5a53643d18369c280323a3bea)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:34.113554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:34.164793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7cc5a5a53643d18369c280323a3bea', 'request_id': 'req-33d93a98-3423-43f4-988b-84542b220e11'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:34.171134 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7cc5a5a53643d18369c280323a3bea', 'request_id': 'req-33d93a98-3423-43f4-988b-84542b220e11'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:34.178827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7cc5a5a53643d18369c280323a3bea', 'request_id': 'req-33d93a98-3423-43f4-988b-84542b220e11'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:34.195578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] The token cache is being invalidate because project 5b7cc5a5a53643d18369c280323a3bea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:35:34.195898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:34.196507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:34.198443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 881/1771] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:35:34 2022] DELETE /identity/v3/projects/5b7cc5a5a53643d18369c280323a3bea => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:34.202215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ee645-9533-4e9c-a646-938f8f82b5b0 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:34.217799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ee645-9533-4e9c-a646-938f8f82b5b0 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:34.280554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115fc8a-7486-4352-b004-59e926b6075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:34.281550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115fc8a-7486-4352-b004-59e926b6075c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:34.281899 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115fc8a-7486-4352-b004-59e926b6075c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:34.282238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115fc8a-7486-4352-b004-59e926b6075c admin admin] PATH_INFO: `/v3/users/fd6b02aff3724b0d8dd0cc85b50f5eaa` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:34.283580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115fc8a-7486-4352-b004-59e926b6075c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd6b02aff3724b0d8dd0cc85b50f5eaa)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:34.285708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115fc8a-7486-4352-b004-59e926b6075c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:34.309919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:34.337418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f115fc8a-7486-4352-b004-59e926b6075c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd6b02aff3724b0d8dd0cc85b50f5eaa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:34.338046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f115fc8a-7486-4352-b004-59e926b6075c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd6b02aff3724b0d8dd0cc85b50f5eaa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:34.350077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f115fc8a-7486-4352-b004-59e926b6075c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd6b02aff3724b0d8dd0cc85b50f5eaa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:34.350793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f115fc8a-7486-4352-b004-59e926b6075c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd6b02aff3724b0d8dd0cc85b50f5eaa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:34.360507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f115fc8a-7486-4352-b004-59e926b6075c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd6b02aff3724b0d8dd0cc85b50f5eaa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:34.378527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 891/1772] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:35:34 2022] DELETE /identity/v3/users/fd6b02aff3724b0d8dd0cc85b50f5eaa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:34.841243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:34.842729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:34.857139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d93a98-3423-43f4-988b-84542b220e11 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:34.867732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:34.919603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:34.921047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:34.921344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:34.921733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] PATH_INFO: `/v3/projects/c38027e126fe4e7cb669cd2bfc0eb290` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:34.923010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c38027e126fe4e7cb669cd2bfc0eb290)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:34.925311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:34.997454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c38027e126fe4e7cb669cd2bfc0eb290', 'request_id': 'req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:35.007403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c38027e126fe4e7cb669cd2bfc0eb290', 'request_id': 'req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:35.018319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c38027e126fe4e7cb669cd2bfc0eb290', 'request_id': 'req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:35.052508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] The token cache is being invalidate because project c38027e126fe4e7cb669cd2bfc0eb290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:35:35.052933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:35.054017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-0c4a0a95-3c7a-46a9-bf22-f03a856248dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:35.058769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 882/1773] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:35:34 2022] DELETE /identity/v3/projects/c38027e126fe4e7cb669cd2bfc0eb290 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:35.061411 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d42c48-fe59-4bdd-a5e8-616067e9fcab None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:35.061703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d42c48-fe59-4bdd-a5e8-616067e9fcab None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:35.061978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d42c48-fe59-4bdd-a5e8-616067e9fcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:35.092717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c6d42c48-fe59-4bdd-a5e8-616067e9fcab None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:35.094264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c6d42c48-fe59-4bdd-a5e8-616067e9fcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:35.185933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 892/1774] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:35.194194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410fc3d-bc9f-49b1-a570-3bfbc540c424 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:35.194610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410fc3d-bc9f-49b1-a570-3bfbc540c424 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:35.194610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410fc3d-bc9f-49b1-a570-3bfbc540c424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:35.214351 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f410fc3d-bc9f-49b1-a570-3bfbc540c424 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:35.215981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f410fc3d-bc9f-49b1-a570-3bfbc540c424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:35.224638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 883/1775] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:35.228922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d42c48-fe59-4bdd-a5e8-616067e9fcab None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:35.239418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d42c48-fe59-4bdd-a5e8-616067e9fcab None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:35.244812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cef0a0-90fd-4e66-866a-9ba0029052d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:35.245494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef0a0-90fd-4e66-866a-9ba0029052d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:35.245782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef0a0-90fd-4e66-866a-9ba0029052d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:35.246061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef0a0-90fd-4e66-866a-9ba0029052d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:35.246841 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cef0a0-90fd-4e66-866a-9ba0029052d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:35.247154 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cef0a0-90fd-4e66-866a-9ba0029052d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:35.248784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cef0a0-90fd-4e66-866a-9ba0029052d9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:35.262665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 893/1776] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:35.267278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410fc3d-bc9f-49b1-a570-3bfbc540c424 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:35.276846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410fc3d-bc9f-49b1-a570-3bfbc540c424 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:35.281823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2af09-a80d-4be8-b0d1-1b9735f3d6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:35.282524 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2af09-a80d-4be8-b0d1-1b9735f3d6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:35.282829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2af09-a80d-4be8-b0d1-1b9735f3d6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:35.283110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2af09-a80d-4be8-b0d1-1b9735f3d6b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:35.283828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2af09-a80d-4be8-b0d1-1b9735f3d6b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:35.285895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2af09-a80d-4be8-b0d1-1b9735f3d6b7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:35.313696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 884/1777] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:35 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:35.318399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cef0a0-90fd-4e66-866a-9ba0029052d9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:35.348831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cef0a0-90fd-4e66-866a-9ba0029052d9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:35.431786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745163f-ebf7-4e04-8293-903e81c140b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:35.432749 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745163f-ebf7-4e04-8293-903e81c140b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:35.433127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745163f-ebf7-4e04-8293-903e81c140b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:35.433482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745163f-ebf7-4e04-8293-903e81c140b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:35.434317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745163f-ebf7-4e04-8293-903e81c140b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:35.441585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745163f-ebf7-4e04-8293-903e81c140b1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:35.465370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 894/1778] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:35 2022] POST /identity/v3/users => generated 297 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:35:35.469447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2af09-a80d-4be8-b0d1-1b9735f3d6b7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:35.481154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2af09-a80d-4be8-b0d1-1b9735f3d6b7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:35.485900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee072be-468f-4fc4-b9d0-7f5fa58535a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:35.486618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee072be-468f-4fc4-b9d0-7f5fa58535a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:35.486936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee072be-468f-4fc4-b9d0-7f5fa58535a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:35.487279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee072be-468f-4fc4-b9d0-7f5fa58535a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:35.488775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee072be-468f-4fc4-b9d0-7f5fa58535a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:35.490409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee072be-468f-4fc4-b9d0-7f5fa58535a1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:35.501918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 885/1779] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:35.506641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745163f-ebf7-4e04-8293-903e81c140b1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:35.517116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745163f-ebf7-4e04-8293-903e81c140b1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:35.523021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d6273-b714-4796-a085-72143fe94b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:35.523825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d6273-b714-4796-a085-72143fe94b7e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:35.524194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d6273-b714-4796-a085-72143fe94b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:35.524542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d6273-b714-4796-a085-72143fe94b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:35.525364 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d6273-b714-4796-a085-72143fe94b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:35.527092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d6273-b714-4796-a085-72143fe94b7e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:35.547197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 895/1780] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:35.548711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee072be-468f-4fc4-b9d0-7f5fa58535a1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:35.560681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee072be-468f-4fc4-b9d0-7f5fa58535a1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:35.565626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65f6e7-4485-461d-853e-867ed4b197d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:35.566535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65f6e7-4485-461d-853e-867ed4b197d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:35.566885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65f6e7-4485-461d-853e-867ed4b197d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:35.567170 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65f6e7-4485-461d-853e-867ed4b197d6 admin admin] PATH_INFO: `/v3/projects/c256a1853054445db979ef05c90acd95/users/34a64361e61749cc9511e83f042274ac/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:35.583239 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65f6e7-4485-461d-853e-867ed4b197d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c256a1853054445db979ef05c90acd95, user_id=34a64361e61749cc9511e83f042274ac, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:35.585916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65f6e7-4485-461d-853e-867ed4b197d6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:35.602340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 886/1781] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:35 2022] PUT /identity/v3/projects/c256a1853054445db979ef05c90acd95/users/34a64361e61749cc9511e83f042274ac/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:35.614581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2a318-94b0-403b-b1f6-cb6d4c75b59c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:35.615193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2a318-94b0-403b-b1f6-cb6d4c75b59c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:35.615620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2a318-94b0-403b-b1f6-cb6d4c75b59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:35.630508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:35.655703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-35b2a318-94b0-403b-b1f6-cb6d4c75b59c None None] MFA Rules not processed for user `34a64361e61749cc9511e83f042274ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:35.656960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-35b2a318-94b0-403b-b1f6-cb6d4c75b59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:35.745248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 896/1782] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:35.750209 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65f6e7-4485-461d-853e-867ed4b197d6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:35.805420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:35.828799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65f6e7-4485-461d-853e-867ed4b197d6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:35.838679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659da06-86b3-41e0-8ebb-9e14ebb32c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:35.840010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659da06-86b3-41e0-8ebb-9e14ebb32c7a admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:35.840391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659da06-86b3-41e0-8ebb-9e14ebb32c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:35.840905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659da06-86b3-41e0-8ebb-9e14ebb32c7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:35.841964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659da06-86b3-41e0-8ebb-9e14ebb32c7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:35.844288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659da06-86b3-41e0-8ebb-9e14ebb32c7a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:35.914014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 887/1783] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:35 2022] POST /identity/v3/projects => generated 393 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:35:35.917237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2a318-94b0-403b-b1f6-cb6d4c75b59c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:35.930301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2a318-94b0-403b-b1f6-cb6d4c75b59c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:36.020638 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35297675-5745-47c3-a1c0-b5c5ab7b5ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:36.021379 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35297675-5745-47c3-a1c0-b5c5ab7b5ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:36.021657 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35297675-5745-47c3-a1c0-b5c5ab7b5ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:36.022535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35297675-5745-47c3-a1c0-b5c5ab7b5ad0 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:36.022640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35297675-5745-47c3-a1c0-b5c5ab7b5ad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:36.024773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35297675-5745-47c3-a1c0-b5c5ab7b5ad0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:36.062498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 897/1784] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:35 2022] POST /identity/v3/users => generated 303 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:36.067368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659da06-86b3-41e0-8ebb-9e14ebb32c7a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:36.080590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659da06-86b3-41e0-8ebb-9e14ebb32c7a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:36.086295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7198886-695b-4f7f-9f92-dc0a696454bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:36.087029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198886-695b-4f7f-9f92-dc0a696454bc admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:36.087317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198886-695b-4f7f-9f92-dc0a696454bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:36.087598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7198886-695b-4f7f-9f92-dc0a696454bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:36.092054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7198886-695b-4f7f-9f92-dc0a696454bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:36.094173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7198886-695b-4f7f-9f92-dc0a696454bc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:36.121390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 888/1785] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:36.122026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35297675-5745-47c3-a1c0-b5c5ab7b5ad0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:36.138042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35297675-5745-47c3-a1c0-b5c5ab7b5ad0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:36.143991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e8656-bd7f-4318-a737-8d948750c721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:36.144978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e8656-bd7f-4318-a737-8d948750c721 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:36.145702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e8656-bd7f-4318-a737-8d948750c721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:36.146155 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e8656-bd7f-4318-a737-8d948750c721 admin admin] PATH_INFO: `/v3/projects/71d2b400968e420587e9882958379f3f/users/9fa9303d12b84d33858e318d37155b98/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:36.172159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e8656-bd7f-4318-a737-8d948750c721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d2b400968e420587e9882958379f3f, user_id=9fa9303d12b84d33858e318d37155b98, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:36.172159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e8656-bd7f-4318-a737-8d948750c721 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:36.191597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 898/1786] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:36 2022] PUT /identity/v3/projects/71d2b400968e420587e9882958379f3f/users/9fa9303d12b84d33858e318d37155b98/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:36.195814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7198886-695b-4f7f-9f92-dc0a696454bc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:36.208964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7198886-695b-4f7f-9f92-dc0a696454bc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:36.293779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a288dc01-2158-4a45-8575-e1ea8d1ffe1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:36.294392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288dc01-2158-4a45-8575-e1ea8d1ffe1d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:36.294669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288dc01-2158-4a45-8575-e1ea8d1ffe1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:36.294992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a288dc01-2158-4a45-8575-e1ea8d1ffe1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:36.295730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288dc01-2158-4a45-8575-e1ea8d1ffe1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:36.297343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a288dc01-2158-4a45-8575-e1ea8d1ffe1d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:36.308400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 889/1787] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:36.313187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e8656-bd7f-4318-a737-8d948750c721 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:36.325935 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e8656-bd7f-4318-a737-8d948750c721 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:36.333030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393445f8-45ca-4004-9017-3977da84c7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:36.334292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393445f8-45ca-4004-9017-3977da84c7ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:36.336700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393445f8-45ca-4004-9017-3977da84c7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:36.337227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393445f8-45ca-4004-9017-3977da84c7ae admin admin] PATH_INFO: `/v3/projects/71d2b400968e420587e9882958379f3f/users/9fa9303d12b84d33858e318d37155b98/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:36.341479 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393445f8-45ca-4004-9017-3977da84c7ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=71d2b400968e420587e9882958379f3f, user_id=9fa9303d12b84d33858e318d37155b98, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:36.344443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393445f8-45ca-4004-9017-3977da84c7ae admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:36.363045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 899/1788] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:36 2022] PUT /identity/v3/projects/71d2b400968e420587e9882958379f3f/users/9fa9303d12b84d33858e318d37155b98/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:36.371750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354ce55-a380-4374-bacc-c212491deb9d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:36.372100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354ce55-a380-4374-bacc-c212491deb9d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:36.372319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354ce55-a380-4374-bacc-c212491deb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:36.405639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a354ce55-a380-4374-bacc-c212491deb9d None None] MFA Rules not processed for user `9fa9303d12b84d33858e318d37155b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:36.406891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a354ce55-a380-4374-bacc-c212491deb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:36.519545 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 890/1789] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:36.530613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393445f8-45ca-4004-9017-3977da84c7ae admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:36.532455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-393445f8-45ca-4004-9017-3977da84c7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:36.592671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:36.599339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393445f8-45ca-4004-9017-3977da84c7ae admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:36.604060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb825b-c064-4fb4-97de-311a0fa4dd0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:36.605030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb825b-c064-4fb4-97de-311a0fa4dd0c service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:36.605304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb825b-c064-4fb4-97de-311a0fa4dd0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:36.605645 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb825b-c064-4fb4-97de-311a0fa4dd0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:36.616230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb825b-c064-4fb4-97de-311a0fa4dd0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:36.617732 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb825b-c064-4fb4-97de-311a0fa4dd0c service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:36.627420 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 900/1790] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:36.682846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354ce55-a380-4374-bacc-c212491deb9d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:36.684512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a354ce55-a380-4374-bacc-c212491deb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:36.698760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354ce55-a380-4374-bacc-c212491deb9d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:36.707587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594903b9-268b-411d-ab43-61d29121018a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:36.709146 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594903b9-268b-411d-ab43-61d29121018a service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:36.709584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594903b9-268b-411d-ab43-61d29121018a service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:36.710074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594903b9-268b-411d-ab43-61d29121018a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:36.757937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:36.801366 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594903b9-268b-411d-ab43-61d29121018a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:36.803796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594903b9-268b-411d-ab43-61d29121018a service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:36.819144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 891/1791] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:46.623141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:46.810050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=12 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:48.907474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bb4e2-28fa-4b62-9315-a4e449c296bf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:48.908017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bb4e2-28fa-4b62-9315-a4e449c296bf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:48.908099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3bb4e2-28fa-4b62-9315-a4e449c296bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:48.940615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ce3bb4e2-28fa-4b62-9315-a4e449c296bf None None] MFA Rules not processed for user `49a1439029fa4b719bc114c3f59e0050`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:48.945343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ce3bb4e2-28fa-4b62-9315-a4e449c296bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:49.023256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 901/1792] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Jun 29 08:35:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:35:49.058873 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bb4e2-28fa-4b62-9315-a4e449c296bf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:49.061167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594903b9-268b-411d-ab43-61d29121018a service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:49.062955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-594903b9-268b-411d-ab43-61d29121018a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:49.064917 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ce3bb4e2-28fa-4b62-9315-a4e449c296bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:49.080229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3bb4e2-28fa-4b62-9315-a4e449c296bf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:49.100198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594903b9-268b-411d-ab43-61d29121018a service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:49.104609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba4d5a-3437-43e0-b1aa-5d20bc82317d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:49.126757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba4d5a-3437-43e0-b1aa-5d20bc82317d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:49.127107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba4d5a-3437-43e0-b1aa-5d20bc82317d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:49.127392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba4d5a-3437-43e0-b1aa-5d20bc82317d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:49.142476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fb4ba-4d55-45ac-a4d6-967acb445e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:49.143264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fb4ba-4d55-45ac-a4d6-967acb445e68 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:49.143552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fb4ba-4d55-45ac-a4d6-967acb445e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:49.145418 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698fb4ba-4d55-45ac-a4d6-967acb445e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:49.166854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698fb4ba-4d55-45ac-a4d6-967acb445e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:49.168375 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698fb4ba-4d55-45ac-a4d6-967acb445e68 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:49.172551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba4d5a-3437-43e0-b1aa-5d20bc82317d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:49.174336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ba4d5a-3437-43e0-b1aa-5d20bc82317d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:49.186596 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 892/1793] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:49.187535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fb4ba-4d55-45ac-a4d6-967acb445e68 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:49.198193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 902/1794] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:49.199083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba4d5a-3437-43e0-b1aa-5d20bc82317d service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:49.200386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-698fb4ba-4d55-45ac-a4d6-967acb445e68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:49.233682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698fb4ba-4d55-45ac-a4d6-967acb445e68 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:49.235103 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba4d5a-3437-43e0-b1aa-5d20bc82317d service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:49.246773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73154537-e5ff-43db-ad41-412edbfef9aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:49.248089 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73154537-e5ff-43db-ad41-412edbfef9aa service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:49.248377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73154537-e5ff-43db-ad41-412edbfef9aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:49.248652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73154537-e5ff-43db-ad41-412edbfef9aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:49.249910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f6659-dc25-4ec3-bf3c-927ec745395c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:49.250570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f6659-dc25-4ec3-bf3c-927ec745395c service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:49.252784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f6659-dc25-4ec3-bf3c-927ec745395c service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:49.253084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f6659-dc25-4ec3-bf3c-927ec745395c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:49.276414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73154537-e5ff-43db-ad41-412edbfef9aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:49.280259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f6659-dc25-4ec3-bf3c-927ec745395c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:49.283454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73154537-e5ff-43db-ad41-412edbfef9aa service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:49.285954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f6659-dc25-4ec3-bf3c-927ec745395c service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:49.312399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 903/1795] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:49.312768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 893/1796] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:50.921369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f6659-dc25-4ec3-bf3c-927ec745395c service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:50.932618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f6659-dc25-4ec3-bf3c-927ec745395c service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:50.945034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee13ce1-bde4-4a7f-bb62-81d81c492135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:50.945034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee13ce1-bde4-4a7f-bb62-81d81c492135 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:50.945034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee13ce1-bde4-4a7f-bb62-81d81c492135 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:50.945034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee13ce1-bde4-4a7f-bb62-81d81c492135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:50.949848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5ee13ce1-bde4-4a7f-bb62-81d81c492135 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:51.107438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee13ce1-bde4-4a7f-bb62-81d81c492135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:51.109177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee13ce1-bde4-4a7f-bb62-81d81c492135 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:51.126926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 904/1797] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:51.834073 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68337cc-c318-40f7-8a4b-e633e7d0ded8 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:51.834835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68337cc-c318-40f7-8a4b-e633e7d0ded8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:51.835193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68337cc-c318-40f7-8a4b-e633e7d0ded8 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:51.837330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 894/1798] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:35:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:35:51.863378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac23f3-49c7-4924-875d-81a80ef4ffe2 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:51.863744 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac23f3-49c7-4924-875d-81a80ef4ffe2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:51.864081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac23f3-49c7-4924-875d-81a80ef4ffe2 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:51.868541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 905/1799] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:35:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:35:51.922250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8cf52-d585-4835-9741-866a4c381862 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:51.922820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8cf52-d585-4835-9741-866a4c381862 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:51.923257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8cf52-d585-4835-9741-866a4c381862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:51.966679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ebf8cf52-d585-4835-9741-866a4c381862 None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:51.969466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ebf8cf52-d585-4835-9741-866a4c381862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:52.039934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe5c6b-7674-4b9e-a5a4-3b34ba62bb6f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:52.040831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe5c6b-7674-4b9e-a5a4-3b34ba62bb6f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:52.044591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe5c6b-7674-4b9e-a5a4-3b34ba62bb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:52.092716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c2fe5c6b-7674-4b9e-a5a4-3b34ba62bb6f None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:52.094409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c2fe5c6b-7674-4b9e-a5a4-3b34ba62bb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:52.162185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 895/1800] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:35:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:52.163441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8cf52-d585-4835-9741-866a4c381862 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:52.259345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 906/1801] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:35:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:35:52.275583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8cf52-d585-4835-9741-866a4c381862 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:52.283619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:52.284544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:52.284861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:52.285154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] PATH_INFO: `/v3/users/34a64361e61749cc9511e83f042274ac` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:52.288325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34a64361e61749cc9511e83f042274ac)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:52.290155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:52.376674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe5c6b-7674-4b9e-a5a4-3b34ba62bb6f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:52.379151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c2fe5c6b-7674-4b9e-a5a4-3b34ba62bb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:52.395437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fe5c6b-7674-4b9e-a5a4-3b34ba62bb6f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:52.404590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34a64361e61749cc9511e83f042274ac'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:52.405515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34a64361e61749cc9511e83f042274ac'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:52.408763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c22ddb-6fdf-4e03-832e-66fd7554ea62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:52.409448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c22ddb-6fdf-4e03-832e-66fd7554ea62 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:52.409742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c22ddb-6fdf-4e03-832e-66fd7554ea62 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:52.410024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c22ddb-6fdf-4e03-832e-66fd7554ea62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:52.414315 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-14c22ddb-6fdf-4e03-832e-66fd7554ea62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:52.426476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34a64361e61749cc9511e83f042274ac'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:52.427826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34a64361e61749cc9511e83f042274ac'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:52.431934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:52.432752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:52.438796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c22ddb-6fdf-4e03-832e-66fd7554ea62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:52.440522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c22ddb-6fdf-4e03-832e-66fd7554ea62 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:52.448255 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34a64361e61749cc9511e83f042274ac'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:52.465696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 907/1802] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:35:52.474906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 896/1803] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:35:52 2022] DELETE /identity/v3/users/34a64361e61749cc9511e83f042274ac => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:52.479746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296ae77-f2a4-4d53-a639-3f14320e3404 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:52.480033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296ae77-f2a4-4d53-a639-3f14320e3404 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:52.480311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296ae77-f2a4-4d53-a639-3f14320e3404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:52.520129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d296ae77-f2a4-4d53-a639-3f14320e3404 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:52.521545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d296ae77-f2a4-4d53-a639-3f14320e3404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:52.561964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:52.563625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:52.616271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 908/1804] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:52.628318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296ae77-f2a4-4d53-a639-3f14320e3404 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:52.630578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d296ae77-f2a4-4d53-a639-3f14320e3404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:52.666675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba5af5-558f-4eb7-a0aa-bc9f7b42a241 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:52.674369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3e284-dc36-4dd5-822b-fdefb746a429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:52.675071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3e284-dc36-4dd5-822b-fdefb746a429 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:52.675356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3e284-dc36-4dd5-822b-fdefb746a429 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:52.675627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3e284-dc36-4dd5-822b-fdefb746a429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:52.678062 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-97f3e284-dc36-4dd5-822b-fdefb746a429 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:52.693546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3e284-dc36-4dd5-822b-fdefb746a429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:52.693746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296ae77-f2a4-4d53-a639-3f14320e3404 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:52.695135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3e284-dc36-4dd5-822b-fdefb746a429 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:52.697535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5f5e7-d589-47cf-bbb3-3d1944d220c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:52.698198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f5e7-d589-47cf-bbb3-3d1944d220c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:52.698480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f5e7-d589-47cf-bbb3-3d1944d220c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:52.698785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c5f5e7-d589-47cf-bbb3-3d1944d220c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:52.708299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 897/1805] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:52.709344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5f5e7-d589-47cf-bbb3-3d1944d220c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:52.710835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c5f5e7-d589-47cf-bbb3-3d1944d220c3 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:52.725068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 909/1806] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:35:53.002187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3e284-dc36-4dd5-822b-fdefb746a429 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:53.004247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-97f3e284-dc36-4dd5-822b-fdefb746a429 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:53.020639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3e284-dc36-4dd5-822b-fdefb746a429 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:53.027168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:53.028144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:53.028350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:53.028622 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] PATH_INFO: `/v3/projects/c256a1853054445db979ef05c90acd95` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:53.029941 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c256a1853054445db979ef05c90acd95)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:53.031824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:53.082659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c256a1853054445db979ef05c90acd95', 'request_id': 'req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:53.089221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c256a1853054445db979ef05c90acd95', 'request_id': 'req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:53.095483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c256a1853054445db979ef05c90acd95', 'request_id': 'req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:53.109551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] The token cache is being invalidate because project c256a1853054445db979ef05c90acd95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:35:53.109834 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:53.110477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:53.112131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 898/1807] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:35:53 2022] DELETE /identity/v3/projects/c256a1853054445db979ef05c90acd95 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:53.116248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5f5e7-d589-47cf-bbb3-3d1944d220c3 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:53.126176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c5f5e7-d589-47cf-bbb3-3d1944d220c3 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:53.186858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66bcd7-e19e-42b9-a8d0-7d52edced641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:53.187730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66bcd7-e19e-42b9-a8d0-7d52edced641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:53.188009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66bcd7-e19e-42b9-a8d0-7d52edced641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:53.188278 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66bcd7-e19e-42b9-a8d0-7d52edced641 admin admin] PATH_INFO: `/v3/users/9fa9303d12b84d33858e318d37155b98` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:53.189495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66bcd7-e19e-42b9-a8d0-7d52edced641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fa9303d12b84d33858e318d37155b98)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:53.191457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66bcd7-e19e-42b9-a8d0-7d52edced641 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:53.242848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:53.248991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ff66bcd7-e19e-42b9-a8d0-7d52edced641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fa9303d12b84d33858e318d37155b98'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:53.249666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ff66bcd7-e19e-42b9-a8d0-7d52edced641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fa9303d12b84d33858e318d37155b98'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:53.262807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ff66bcd7-e19e-42b9-a8d0-7d52edced641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fa9303d12b84d33858e318d37155b98'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:53.263428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ff66bcd7-e19e-42b9-a8d0-7d52edced641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fa9303d12b84d33858e318d37155b98'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:53.272306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ff66bcd7-e19e-42b9-a8d0-7d52edced641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fa9303d12b84d33858e318d37155b98'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:53.285361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 910/1808] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:35:53 2022] DELETE /identity/v3/users/9fa9303d12b84d33858e318d37155b98 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:53.849994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:53.852085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:53.867235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705d4c8-5165-4cbc-9db3-cfe56e5230ec admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:53.927654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:53.937583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:53.938838 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:53.939439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:53.939809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] PATH_INFO: `/v3/projects/71d2b400968e420587e9882958379f3f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:53.941170 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71d2b400968e420587e9882958379f3f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:53.943216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:53.991341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71d2b400968e420587e9882958379f3f', 'request_id': 'req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:53.997810 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71d2b400968e420587e9882958379f3f', 'request_id': 'req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:54.003248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71d2b400968e420587e9882958379f3f', 'request_id': 'req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:54.017953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] The token cache is being invalidate because project 71d2b400968e420587e9882958379f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:35:54.018256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:54.018933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fdcfb77a-0f4f-4fdd-8b2f-589550424b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:35:54.020760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 899/1809] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:35:53 2022] DELETE /identity/v3/projects/71d2b400968e420587e9882958379f3f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:54.025681 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163c02b-b6f3-4615-8819-a02d9d5d21fb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.025970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163c02b-b6f3-4615-8819-a02d9d5d21fb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.026248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163c02b-b6f3-4615-8819-a02d9d5d21fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.049039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3163c02b-b6f3-4615-8819-a02d9d5d21fb None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:54.050864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3163c02b-b6f3-4615-8819-a02d9d5d21fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:54.115537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 911/1810] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:54.129819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f606432-fd8f-42ae-a396-0966123a56ad None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.130214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f606432-fd8f-42ae-a396-0966123a56ad None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.130645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f606432-fd8f-42ae-a396-0966123a56ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.151963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5f606432-fd8f-42ae-a396-0966123a56ad None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:54.153176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5f606432-fd8f-42ae-a396-0966123a56ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:54.162669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 900/1811] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:54.168053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163c02b-b6f3-4615-8819-a02d9d5d21fb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:54.185255 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163c02b-b6f3-4615-8819-a02d9d5d21fb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:54.197824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977eb37-1015-4111-88a9-ff191588d8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:54.198490 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977eb37-1015-4111-88a9-ff191588d8fe admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.198835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977eb37-1015-4111-88a9-ff191588d8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.199104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977eb37-1015-4111-88a9-ff191588d8fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.199837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977eb37-1015-4111-88a9-ff191588d8fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:35:54.200139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977eb37-1015-4111-88a9-ff191588d8fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:54.201763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977eb37-1015-4111-88a9-ff191588d8fe admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:54.216222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 912/1812] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:35:54.222845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f606432-fd8f-42ae-a396-0966123a56ad None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:54.232697 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f606432-fd8f-42ae-a396-0966123a56ad None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:54.238493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d946865-1121-44b1-a60c-6da675858af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:54.239191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d946865-1121-44b1-a60c-6da675858af2 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.239474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d946865-1121-44b1-a60c-6da675858af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.239753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d946865-1121-44b1-a60c-6da675858af2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.240454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d946865-1121-44b1-a60c-6da675858af2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:54.242405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d946865-1121-44b1-a60c-6da675858af2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:54.270855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 901/1813] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:54 2022] POST /identity/v3/projects => generated 407 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:54.275288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977eb37-1015-4111-88a9-ff191588d8fe admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:54.285745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977eb37-1015-4111-88a9-ff191588d8fe admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:54.348886 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a08ec-e5fb-4148-93d5-32dbef685cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:54.349976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a08ec-e5fb-4148-93d5-32dbef685cef admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.350437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a08ec-e5fb-4148-93d5-32dbef685cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.350940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a08ec-e5fb-4148-93d5-32dbef685cef admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.352950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a08ec-e5fb-4148-93d5-32dbef685cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:54.356379 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a08ec-e5fb-4148-93d5-32dbef685cef admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:54.386901 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 913/1814] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:54 2022] POST /identity/v3/users => generated 304 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:54.391446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d946865-1121-44b1-a60c-6da675858af2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:54.404677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d946865-1121-44b1-a60c-6da675858af2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:54.412985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6d4c2-9708-40bd-8f39-485e598c9c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:54.413856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6d4c2-9708-40bd-8f39-485e598c9c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.414274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6d4c2-9708-40bd-8f39-485e598c9c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.414573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6d4c2-9708-40bd-8f39-485e598c9c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.415589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6d4c2-9708-40bd-8f39-485e598c9c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:54.417748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6d4c2-9708-40bd-8f39-485e598c9c02 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:54.433354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 902/1815] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:54.437510 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a08ec-e5fb-4148-93d5-32dbef685cef admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:54.447970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a08ec-e5fb-4148-93d5-32dbef685cef admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:54.453695 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7058d5-3b4b-4e99-87cc-9fab6f76efa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:54.454372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7058d5-3b4b-4e99-87cc-9fab6f76efa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.454631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7058d5-3b4b-4e99-87cc-9fab6f76efa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.455022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7058d5-3b4b-4e99-87cc-9fab6f76efa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.455761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7058d5-3b4b-4e99-87cc-9fab6f76efa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:54.457405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7058d5-3b4b-4e99-87cc-9fab6f76efa6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:54.468555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 914/1816] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:35:54.472918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6d4c2-9708-40bd-8f39-485e598c9c02 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:54.483473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6d4c2-9708-40bd-8f39-485e598c9c02 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:54.489333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dc5c5-b8f5-42c1-9bfa-9b1f7d26b50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:54.490225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dc5c5-b8f5-42c1-9bfa-9b1f7d26b50a admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.490458 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dc5c5-b8f5-42c1-9bfa-9b1f7d26b50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.490791 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201dc5c5-b8f5-42c1-9bfa-9b1f7d26b50a admin admin] PATH_INFO: `/v3/projects/accc84588f1649579b3ba1ff90650475/users/dc35b11901d547dcbf2385a522dcfcf3/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.506621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dc5c5-b8f5-42c1-9bfa-9b1f7d26b50a admin admin] RBAC: Authorizing `identity:create_grant(project_id=accc84588f1649579b3ba1ff90650475, user_id=dc35b11901d547dcbf2385a522dcfcf3, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:54.510682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201dc5c5-b8f5-42c1-9bfa-9b1f7d26b50a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:54.529262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 903/1817] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:54 2022] PUT /identity/v3/projects/accc84588f1649579b3ba1ff90650475/users/dc35b11901d547dcbf2385a522dcfcf3/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:54.539708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cec14-87ff-4a29-9696-c39fb053a77d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.539998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cec14-87ff-4a29-9696-c39fb053a77d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.540273 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cec14-87ff-4a29-9696-c39fb053a77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.568859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b66cec14-87ff-4a29-9696-c39fb053a77d None None] MFA Rules not processed for user `dc35b11901d547dcbf2385a522dcfcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:54.570058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b66cec14-87ff-4a29-9696-c39fb053a77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:54.582621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:54.639783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 915/1818] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:54 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:54.644356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dc5c5-b8f5-42c1-9bfa-9b1f7d26b50a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:54.712919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201dc5c5-b8f5-42c1-9bfa-9b1f7d26b50a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:54.717786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105e443-a41e-4382-aa88-76f9c4c03527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:54.718460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105e443-a41e-4382-aa88-76f9c4c03527 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.718770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105e443-a41e-4382-aa88-76f9c4c03527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.719048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105e443-a41e-4382-aa88-76f9c4c03527 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.719787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105e443-a41e-4382-aa88-76f9c4c03527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:54.721707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105e443-a41e-4382-aa88-76f9c4c03527 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:54.754971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 904/1819] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:35:54 2022] POST /identity/v3/projects => generated 409 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:54.756188 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cec14-87ff-4a29-9696-c39fb053a77d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:54.767401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cec14-87ff-4a29-9696-c39fb053a77d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:54.843808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857e257-c793-46b4-aa17-97919122d3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:54.844498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857e257-c793-46b4-aa17-97919122d3f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.844784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857e257-c793-46b4-aa17-97919122d3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.845058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857e257-c793-46b4-aa17-97919122d3f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.845767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857e257-c793-46b4-aa17-97919122d3f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:54.847737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857e257-c793-46b4-aa17-97919122d3f2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:54.871149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 916/1820] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:35:54 2022] POST /identity/v3/users => generated 311 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:35:54.875098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105e443-a41e-4382-aa88-76f9c4c03527 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:54.881890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:35:54.886033 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105e443-a41e-4382-aa88-76f9c4c03527 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:54.891955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f548d0d-7667-47a6-84f8-4e41abe41ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:54.892842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f548d0d-7667-47a6-84f8-4e41abe41ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.893216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f548d0d-7667-47a6-84f8-4e41abe41ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.893657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f548d0d-7667-47a6-84f8-4e41abe41ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.894554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f548d0d-7667-47a6-84f8-4e41abe41ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:54.896410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f548d0d-7667-47a6-84f8-4e41abe41ce6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:54.908399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 905/1821] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:54.912462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857e257-c793-46b4-aa17-97919122d3f2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:54.923473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857e257-c793-46b4-aa17-97919122d3f2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:54.928548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc69b1c-0935-44a5-86e9-6378d8e30e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:54.929460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc69b1c-0935-44a5-86e9-6378d8e30e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:54.929799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc69b1c-0935-44a5-86e9-6378d8e30e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:54.930079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc69b1c-0935-44a5-86e9-6378d8e30e35 admin admin] PATH_INFO: `/v3/projects/88650ab507644dbbbce797b7ca4f480f/users/0b9145b9234a46aeb6b4a1fcc1501911/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:54.965301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc69b1c-0935-44a5-86e9-6378d8e30e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88650ab507644dbbbce797b7ca4f480f, user_id=0b9145b9234a46aeb6b4a1fcc1501911, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:54.968734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc69b1c-0935-44a5-86e9-6378d8e30e35 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:54.984617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 917/1822] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:54 2022] PUT /identity/v3/projects/88650ab507644dbbbce797b7ca4f480f/users/0b9145b9234a46aeb6b4a1fcc1501911/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:54.989063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f548d0d-7667-47a6-84f8-4e41abe41ce6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:54.999855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f548d0d-7667-47a6-84f8-4e41abe41ce6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:55.073753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a307bded-31e7-4226-b28d-459ae1ce5692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:55.074529 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307bded-31e7-4226-b28d-459ae1ce5692 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:55.074907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307bded-31e7-4226-b28d-459ae1ce5692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:55.075250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307bded-31e7-4226-b28d-459ae1ce5692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:55.076183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307bded-31e7-4226-b28d-459ae1ce5692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:55.077862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a307bded-31e7-4226-b28d-459ae1ce5692 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:55.088627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 906/1823] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:35:55.093174 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc69b1c-0935-44a5-86e9-6378d8e30e35 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:55.103909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc69b1c-0935-44a5-86e9-6378d8e30e35 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:55.114102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5dfda-335e-488d-8483-41b229fe79a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:55.115064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5dfda-335e-488d-8483-41b229fe79a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:55.115353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5dfda-335e-488d-8483-41b229fe79a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:55.115631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5dfda-335e-488d-8483-41b229fe79a7 admin admin] PATH_INFO: `/v3/projects/88650ab507644dbbbce797b7ca4f480f/users/0b9145b9234a46aeb6b4a1fcc1501911/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:55.117664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5dfda-335e-488d-8483-41b229fe79a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88650ab507644dbbbce797b7ca4f480f, user_id=0b9145b9234a46aeb6b4a1fcc1501911, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:55.120519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5dfda-335e-488d-8483-41b229fe79a7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:55.131485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 918/1824] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:35:55 2022] PUT /identity/v3/projects/88650ab507644dbbbce797b7ca4f480f/users/0b9145b9234a46aeb6b4a1fcc1501911/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:35:55.141962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e51ab3-5f55-4583-9b31-5400941a125b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:55.142279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e51ab3-5f55-4583-9b31-5400941a125b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:55.142557 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e51ab3-5f55-4583-9b31-5400941a125b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:55.169031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-19e51ab3-5f55-4583-9b31-5400941a125b None None] MFA Rules not processed for user `0b9145b9234a46aeb6b4a1fcc1501911`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:35:55.170231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-19e51ab3-5f55-4583-9b31-5400941a125b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:55.239395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 907/1825] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:35:55 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:35:55.250339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5dfda-335e-488d-8483-41b229fe79a7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:55.252210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-43f5dfda-335e-488d-8483-41b229fe79a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:35:55.312243 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5dfda-335e-488d-8483-41b229fe79a7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:55.316573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255bf75-995e-40de-ba06-cccd743eb0b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:55.317230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255bf75-995e-40de-ba06-cccd743eb0b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:55.317512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255bf75-995e-40de-ba06-cccd743eb0b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:55.317781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255bf75-995e-40de-ba06-cccd743eb0b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:55.328290 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255bf75-995e-40de-ba06-cccd743eb0b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:55.329729 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255bf75-995e-40de-ba06-cccd743eb0b0 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:55.340027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 919/1826] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:55 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:35:55.519933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e51ab3-5f55-4583-9b31-5400941a125b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:35:55.530359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e51ab3-5f55-4583-9b31-5400941a125b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:35:55.534532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fadfd-4e63-4fe0-ab79-3a24626afe08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:35:55.535209 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fadfd-4e63-4fe0-ab79-3a24626afe08 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:35:55.535491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fadfd-4e63-4fe0-ab79-3a24626afe08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:35:55.535770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fadfd-4e63-4fe0-ab79-3a24626afe08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:35:55.601406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fadfd-4e63-4fe0-ab79-3a24626afe08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:35:55.603068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fadfd-4e63-4fe0-ab79-3a24626afe08 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:35:55.612581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 908/1827] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:35:55 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:05.335336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:05.608394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:20.513584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec8659-5a94-4d77-8dc5-ec58e3fa7a7a None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:20.514198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec8659-5a94-4d77-8dc5-ec58e3fa7a7a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:20.514198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec8659-5a94-4d77-8dc5-ec58e3fa7a7a None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:20.515802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 920/1828] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:36:20.525151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456eb0d-f736-45cd-9b3b-f2844dc6cb1c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:20.525673 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456eb0d-f736-45cd-9b3b-f2844dc6cb1c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:20.525932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456eb0d-f736-45cd-9b3b-f2844dc6cb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:20.547182 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c456eb0d-f736-45cd-9b3b-f2844dc6cb1c None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:20.548416 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c456eb0d-f736-45cd-9b3b-f2844dc6cb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:20.620925 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 909/1829] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:36:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:20.633839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec8659-5a94-4d77-8dc5-ec58e3fa7a7a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:20.635546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-16ec8659-5a94-4d77-8dc5-ec58e3fa7a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:20.654455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec8659-5a94-4d77-8dc5-ec58e3fa7a7a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:20.660441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7bfab-e9d3-4fca-b520-29f9768fb4ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:20.661186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7bfab-e9d3-4fca-b520-29f9768fb4ca service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:20.661546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7bfab-e9d3-4fca-b520-29f9768fb4ca service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:20.662019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7bfab-e9d3-4fca-b520-29f9768fb4ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:20.671748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7bfab-e9d3-4fca-b520-29f9768fb4ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:20.673547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7bfab-e9d3-4fca-b520-29f9768fb4ca service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:20.685257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 921/1830] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:24.027646 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456eb0d-f736-45cd-9b3b-f2844dc6cb1c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:24.029470 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c456eb0d-f736-45cd-9b3b-f2844dc6cb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:24.073045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7bfab-e9d3-4fca-b520-29f9768fb4ca service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:24.130041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456eb0d-f736-45cd-9b3b-f2844dc6cb1c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:24.135609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:24.136603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:24.137106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:24.137544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] PATH_INFO: `/v3/users/6c137b92885547baba30e8005b3c89db` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:24.139018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c137b92885547baba30e8005b3c89db)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:24.141305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:24.160399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7bfab-e9d3-4fca-b520-29f9768fb4ca service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:24.166572 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd7aaa-d0cd-4005-9cd2-6cdb195af57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:24.167984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd7aaa-d0cd-4005-9cd2-6cdb195af57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:24.168863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd7aaa-d0cd-4005-9cd2-6cdb195af57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:24.169778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd7aaa-d0cd-4005-9cd2-6cdb195af57a admin admin] PATH_INFO: `/v3/users/dc35b11901d547dcbf2385a522dcfcf3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:24.172651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd7aaa-d0cd-4005-9cd2-6cdb195af57a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc35b11901d547dcbf2385a522dcfcf3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:24.174952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd7aaa-d0cd-4005-9cd2-6cdb195af57a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:24.212270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c137b92885547baba30e8005b3c89db'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.213199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c137b92885547baba30e8005b3c89db'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.226975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c137b92885547baba30e8005b3c89db'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.227641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c137b92885547baba30e8005b3c89db'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.232368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d9fd7aaa-d0cd-4005-9cd2-6cdb195af57a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc35b11901d547dcbf2385a522dcfcf3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.233191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d9fd7aaa-d0cd-4005-9cd2-6cdb195af57a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc35b11901d547dcbf2385a522dcfcf3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.236200 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c137b92885547baba30e8005b3c89db'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.246477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d9fd7aaa-d0cd-4005-9cd2-6cdb195af57a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc35b11901d547dcbf2385a522dcfcf3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.247185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d9fd7aaa-d0cd-4005-9cd2-6cdb195af57a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc35b11901d547dcbf2385a522dcfcf3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.248874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 910/1831] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:24 2022] DELETE /identity/v3/users/6c137b92885547baba30e8005b3c89db => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:24.252969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:24.254310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:24.256446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d9fd7aaa-d0cd-4005-9cd2-6cdb195af57a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc35b11901d547dcbf2385a522dcfcf3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.268915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:24.272869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 922/1832] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:24 2022] DELETE /identity/v3/users/dc35b11901d547dcbf2385a522dcfcf3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:36:24.274166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c75a20-d1e1-44d0-a29a-3e5ebc53be02 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:24.274458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c75a20-d1e1-44d0-a29a-3e5ebc53be02 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:24.274754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c75a20-d1e1-44d0-a29a-3e5ebc53be02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:24.293582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e2c75a20-d1e1-44d0-a29a-3e5ebc53be02 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:24.294834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e2c75a20-d1e1-44d0-a29a-3e5ebc53be02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:24.324464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8613b85-12e9-4b7b-85d2-cdc6e357d3e8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:24.353850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 923/1833] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:24.355194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354a881-c40c-400a-a924-2e0f0e305b98 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:24.355454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354a881-c40c-400a-a924-2e0f0e305b98 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:24.355739 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354a881-c40c-400a-a924-2e0f0e305b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:24.368045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:24.375744 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5354a881-c40c-400a-a924-2e0f0e305b98 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:24.377893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5354a881-c40c-400a-a924-2e0f0e305b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:24.385764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d758fb6-cc68-4c45-9b8d-b38f0b751621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:24.386248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 924/1834] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:24.386752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d758fb6-cc68-4c45-9b8d-b38f0b751621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:24.387040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d758fb6-cc68-4c45-9b8d-b38f0b751621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:24.387137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354a881-c40c-400a-a924-2e0f0e305b98 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:24.387313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d758fb6-cc68-4c45-9b8d-b38f0b751621 admin admin] PATH_INFO: `/v3/users/e1abb107409a4854ab9d47ff176d84a5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:24.388513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d758fb6-cc68-4c45-9b8d-b38f0b751621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1abb107409a4854ab9d47ff176d84a5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:24.390466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5354a881-c40c-400a-a924-2e0f0e305b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:24.392394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d758fb6-cc68-4c45-9b8d-b38f0b751621 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:24.447616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9d758fb6-cc68-4c45-9b8d-b38f0b751621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1abb107409a4854ab9d47ff176d84a5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.448278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9d758fb6-cc68-4c45-9b8d-b38f0b751621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1abb107409a4854ab9d47ff176d84a5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.461473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354a881-c40c-400a-a924-2e0f0e305b98 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:24.470046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3c5f4-327d-48d5-9d1f-74cc7f331ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:24.470876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3c5f4-327d-48d5-9d1f-74cc7f331ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:24.471257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3c5f4-327d-48d5-9d1f-74cc7f331ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:24.471629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3c5f4-327d-48d5-9d1f-74cc7f331ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:24.473606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-43a3c5f4-327d-48d5-9d1f-74cc7f331ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:24.484358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9d758fb6-cc68-4c45-9b8d-b38f0b751621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1abb107409a4854ab9d47ff176d84a5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.485434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9d758fb6-cc68-4c45-9b8d-b38f0b751621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1abb107409a4854ab9d47ff176d84a5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.490378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3c5f4-327d-48d5-9d1f-74cc7f331ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:24.492201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3c5f4-327d-48d5-9d1f-74cc7f331ba4 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:24.497411 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-9d758fb6-cc68-4c45-9b8d-b38f0b751621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1abb107409a4854ab9d47ff176d84a5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:24.503852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 925/1835] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:24.504607 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3c5f4-327d-48d5-9d1f-74cc7f331ba4 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:24.506014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-43a3c5f4-327d-48d5-9d1f-74cc7f331ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:24.522504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3c5f4-327d-48d5-9d1f-74cc7f331ba4 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:24.534552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 911/1836] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:24 2022] DELETE /identity/v3/users/e1abb107409a4854ab9d47ff176d84a5 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:24.538883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2127c-893f-44dc-a7e3-a8d4d01ee589 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:24.539178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2127c-893f-44dc-a7e3-a8d4d01ee589 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:24.539460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2127c-893f-44dc-a7e3-a8d4d01ee589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:24.541186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a42f63-d9bd-4a84-82a3-e811a6ef807a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:24.541850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a42f63-d9bd-4a84-82a3-e811a6ef807a service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:24.542130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a42f63-d9bd-4a84-82a3-e811a6ef807a service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:24.542402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a42f63-d9bd-4a84-82a3-e811a6ef807a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:24.545460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-21a42f63-d9bd-4a84-82a3-e811a6ef807a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:24.577671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a42f63-d9bd-4a84-82a3-e811a6ef807a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:24.578219 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9ad2127c-893f-44dc-a7e3-a8d4d01ee589 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:24.579553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9ad2127c-893f-44dc-a7e3-a8d4d01ee589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:24.579842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a42f63-d9bd-4a84-82a3-e811a6ef807a service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:24.616687 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 926/1837] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:24.639047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:24.699888 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 912/1838] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:24.725412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a42f63-d9bd-4a84-82a3-e811a6ef807a service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:24.730909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-21a42f63-d9bd-4a84-82a3-e811a6ef807a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:24.749473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a42f63-d9bd-4a84-82a3-e811a6ef807a service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:24.839602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af742d0f-4b9b-4519-812f-4001717307f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:24.840733 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af742d0f-4b9b-4519-812f-4001717307f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:24.840913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af742d0f-4b9b-4519-812f-4001717307f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:24.841164 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af742d0f-4b9b-4519-812f-4001717307f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:24.862971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af742d0f-4b9b-4519-812f-4001717307f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:24.864526 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af742d0f-4b9b-4519-812f-4001717307f3 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:24.879855 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 927/1839] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:24.962342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2127c-893f-44dc-a7e3-a8d4d01ee589 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:24.967375 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9ad2127c-893f-44dc-a7e3-a8d4d01ee589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:25.005206 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2127c-893f-44dc-a7e3-a8d4d01ee589 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:25.023318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:25.025276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:25.029107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:25.029772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] PATH_INFO: `/v3/projects/7eda322434244f0cb317b36b99f5f6ec` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:25.033031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eda322434244f0cb317b36b99f5f6ec)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:25.035567 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:25.110753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eda322434244f0cb317b36b99f5f6ec', 'request_id': 'req-510adab3-418c-46a9-b9e7-6074f2ffb2f1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.133989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eda322434244f0cb317b36b99f5f6ec', 'request_id': 'req-510adab3-418c-46a9-b9e7-6074f2ffb2f1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.146798 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eda322434244f0cb317b36b99f5f6ec', 'request_id': 'req-510adab3-418c-46a9-b9e7-6074f2ffb2f1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.164312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] The token cache is being invalidate because project 7eda322434244f0cb317b36b99f5f6ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:25.164659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.165230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.167424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 913/1840] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:24 2022] DELETE /identity/v3/projects/7eda322434244f0cb317b36b99f5f6ec => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:25.171608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af742d0f-4b9b-4519-812f-4001717307f3 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:25.184361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af742d0f-4b9b-4519-812f-4001717307f3 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:25.247363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:25.248259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:25.248540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:25.248814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] PATH_INFO: `/v3/users/2256a3d1362f4b0891a74707e68e920c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:25.250062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2256a3d1362f4b0891a74707e68e920c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:25.252180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:25.297946 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:25.299826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:25.316153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510adab3-418c-46a9-b9e7-6074f2ffb2f1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:25.321632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:25.323957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:25.325212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:25.325597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:25.326039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] PATH_INFO: `/v3/projects/940ac7c4616941e2890d2e6943407027` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:25.327756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=940ac7c4616941e2890d2e6943407027)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:25.330448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:25.340561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2256a3d1362f4b0891a74707e68e920c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.341247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2256a3d1362f4b0891a74707e68e920c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.357128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2256a3d1362f4b0891a74707e68e920c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.358302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2256a3d1362f4b0891a74707e68e920c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.368682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2256a3d1362f4b0891a74707e68e920c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.385519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 928/1841] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:25 2022] DELETE /identity/v3/users/2256a3d1362f4b0891a74707e68e920c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:25.394604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '940ac7c4616941e2890d2e6943407027', 'request_id': 'req-f1ba3f81-6140-4efc-927b-6b54160931d0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.400764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '940ac7c4616941e2890d2e6943407027', 'request_id': 'req-f1ba3f81-6140-4efc-927b-6b54160931d0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.418048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '940ac7c4616941e2890d2e6943407027', 'request_id': 'req-f1ba3f81-6140-4efc-927b-6b54160931d0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.442095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] The token cache is being invalidate because project 940ac7c4616941e2890d2e6943407027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:25.442452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.443133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.454413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 914/1842] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:25 2022] DELETE /identity/v3/projects/940ac7c4616941e2890d2e6943407027 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:36:25.465886 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:25.467540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:25.483949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:25.485569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:25.555792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:25.558283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba35ad7-b79d-41cc-9aa7-ece08af4e6f8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:25.565786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:25.567571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:25.567571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:25.568013 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] PATH_INFO: `/v3/users/e712bd9b2497416ba1e7b37b74b7c073` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:25.569343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e712bd9b2497416ba1e7b37b74b7c073)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:25.571523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:25.577293 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba3f81-6140-4efc-927b-6b54160931d0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:25.582474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:25.583500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:25.583672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:25.583931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] PATH_INFO: `/v3/projects/accc84588f1649579b3ba1ff90650475` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:25.591054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=accc84588f1649579b3ba1ff90650475)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:25.593169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:25.637793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e712bd9b2497416ba1e7b37b74b7c073'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.638473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e712bd9b2497416ba1e7b37b74b7c073'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.651801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'accc84588f1649579b3ba1ff90650475', 'request_id': 'req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.655752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e712bd9b2497416ba1e7b37b74b7c073'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.657211 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e712bd9b2497416ba1e7b37b74b7c073'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.659505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'accc84588f1649579b3ba1ff90650475', 'request_id': 'req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.667194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'accc84588f1649579b3ba1ff90650475', 'request_id': 'req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.669358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e712bd9b2497416ba1e7b37b74b7c073'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.684423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] The token cache is being invalidate because project accc84588f1649579b3ba1ff90650475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:25.684981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.686067 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.687635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 929/1843] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:25 2022] DELETE /identity/v3/users/e712bd9b2497416ba1e7b37b74b7c073 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:25.688342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 915/1844] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:25 2022] DELETE /identity/v3/projects/accc84588f1649579b3ba1ff90650475 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:25.692581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:25.694048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:25.732259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:25.751513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:25.753840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:25.778786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0d2a0-3912-4404-b011-b82c0540b08c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:25.786912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b2864-5574-4021-9c0d-fbd97d29ca14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:25.789948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b2864-5574-4021-9c0d-fbd97d29ca14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:25.791135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b2864-5574-4021-9c0d-fbd97d29ca14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:25.791772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b2864-5574-4021-9c0d-fbd97d29ca14 admin admin] PATH_INFO: `/v3/users/0b9145b9234a46aeb6b4a1fcc1501911` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:25.802480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b2864-5574-4021-9c0d-fbd97d29ca14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b9145b9234a46aeb6b4a1fcc1501911)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:25.807909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b2864-5574-4021-9c0d-fbd97d29ca14 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:25.855378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44fefa-8ef4-4845-8bab-f12feaf64e8f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:25.861575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:25.862815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:25.862985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:25.863344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] PATH_INFO: `/v3/projects/25657cdd64a844dd869d948094851345` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:25.864707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25657cdd64a844dd869d948094851345)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:25.866816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:25.889860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c12b2864-5574-4021-9c0d-fbd97d29ca14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b9145b9234a46aeb6b4a1fcc1501911'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.890673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c12b2864-5574-4021-9c0d-fbd97d29ca14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b9145b9234a46aeb6b4a1fcc1501911'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.927830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c12b2864-5574-4021-9c0d-fbd97d29ca14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b9145b9234a46aeb6b4a1fcc1501911'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.929150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c12b2864-5574-4021-9c0d-fbd97d29ca14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b9145b9234a46aeb6b4a1fcc1501911'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.942790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c12b2864-5574-4021-9c0d-fbd97d29ca14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b9145b9234a46aeb6b4a1fcc1501911'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.956394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25657cdd64a844dd869d948094851345', 'request_id': 'req-e8a49f86-61b9-4483-89d0-799acd28a3e5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.964457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25657cdd64a844dd869d948094851345', 'request_id': 'req-e8a49f86-61b9-4483-89d0-799acd28a3e5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.971398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 930/1845] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:25 2022] DELETE /identity/v3/users/0b9145b9234a46aeb6b4a1fcc1501911 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:36:25.973056 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25657cdd64a844dd869d948094851345', 'request_id': 'req-e8a49f86-61b9-4483-89d0-799acd28a3e5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.992619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] The token cache is being invalidate because project 25657cdd64a844dd869d948094851345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:25.992961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:25.993719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:26.003114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 916/1846] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:25 2022] DELETE /identity/v3/projects/25657cdd64a844dd869d948094851345 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:36:26.005523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aef45e-bedc-476a-b2fd-9e5591af73e7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.005883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aef45e-bedc-476a-b2fd-9e5591af73e7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.006231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aef45e-bedc-476a-b2fd-9e5591af73e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.034200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f0aef45e-bedc-476a-b2fd-9e5591af73e7 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:26.036626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f0aef45e-bedc-476a-b2fd-9e5591af73e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:26.142027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.144289 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:26.152471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:26.193471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 931/1847] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:36:26.195569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98290704-7a1b-4601-a082-96fec61a7777 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.195999 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98290704-7a1b-4601-a082-96fec61a7777 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.196523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98290704-7a1b-4601-a082-96fec61a7777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.221453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-98290704-7a1b-4601-a082-96fec61a7777 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:26.224427 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-98290704-7a1b-4601-a082-96fec61a7777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:26.240160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 932/1848] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:26.241352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98290704-7a1b-4601-a082-96fec61a7777 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.253709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98290704-7a1b-4601-a082-96fec61a7777 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:26.260492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a181344-94f5-4db4-9504-dadd481c9ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:26.261468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a181344-94f5-4db4-9504-dadd481c9ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.261838 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a181344-94f5-4db4-9504-dadd481c9ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.262187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a181344-94f5-4db4-9504-dadd481c9ed4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.263807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a49f86-61b9-4483-89d0-799acd28a3e5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:26.264308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a181344-94f5-4db4-9504-dadd481c9ed4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:36:26.264999 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a181344-94f5-4db4-9504-dadd481c9ed4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:26.268570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a181344-94f5-4db4-9504-dadd481c9ed4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:26.273019 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:26.274187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.274855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.275453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] PATH_INFO: `/v3/projects/5e12b1a75cd8466b86d317386c0ea980` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.277167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e12b1a75cd8466b86d317386c0ea980)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:26.280090 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:26.288377 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 933/1849] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:36:26.289339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a181344-94f5-4db4-9504-dadd481c9ed4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.300455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a181344-94f5-4db4-9504-dadd481c9ed4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:26.307256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f959e-99ef-47b2-963d-c232a70234ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:26.308362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f959e-99ef-47b2-963d-c232a70234ac admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.308827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f959e-99ef-47b2-963d-c232a70234ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.309198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f959e-99ef-47b2-963d-c232a70234ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.310009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f959e-99ef-47b2-963d-c232a70234ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:26.312871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f959e-99ef-47b2-963d-c232a70234ac admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:26.355595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 934/1850] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:26 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:26.355857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e12b1a75cd8466b86d317386c0ea980', 'request_id': 'req-d40ca81e-4da2-45d0-b0ea-11dd7262343c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:26.359971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f959e-99ef-47b2-963d-c232a70234ac admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.364210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e12b1a75cd8466b86d317386c0ea980', 'request_id': 'req-d40ca81e-4da2-45d0-b0ea-11dd7262343c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:26.375964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e12b1a75cd8466b86d317386c0ea980', 'request_id': 'req-d40ca81e-4da2-45d0-b0ea-11dd7262343c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:26.379897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:26.398030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] The token cache is being invalidate because project 5e12b1a75cd8466b86d317386c0ea980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:26.398664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:26.399260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d40ca81e-4da2-45d0-b0ea-11dd7262343c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:26.404606 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 917/1851] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:26 2022] DELETE /identity/v3/projects/5e12b1a75cd8466b86d317386c0ea980 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:26.406583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ed1a9-b1c6-4cc3-96a7-f44afb07fd43 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.407311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ed1a9-b1c6-4cc3-96a7-f44afb07fd43 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.408683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ed1a9-b1c6-4cc3-96a7-f44afb07fd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.452894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-058ed1a9-b1c6-4cc3-96a7-f44afb07fd43 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:26.456385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-058ed1a9-b1c6-4cc3-96a7-f44afb07fd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:26.467100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f959e-99ef-47b2-963d-c232a70234ac admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:26.480875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 918/1852] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:26.481281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1a46d-e6e0-4981-9c66-317bb339a0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:26.481281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1a46d-e6e0-4981-9c66-317bb339a0db admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.481281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1a46d-e6e0-4981-9c66-317bb339a0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.481281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1a46d-e6e0-4981-9c66-317bb339a0db admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.481586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1a46d-e6e0-4981-9c66-317bb339a0db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:26.481994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1a46d-e6e0-4981-9c66-317bb339a0db admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:26.483474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9084-9676-451f-a31d-48c44cfc9bcd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.483850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9084-9676-451f-a31d-48c44cfc9bcd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.484322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9084-9676-451f-a31d-48c44cfc9bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.508982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-aabf9084-9676-451f-a31d-48c44cfc9bcd None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:26.510266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-aabf9084-9676-451f-a31d-48c44cfc9bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:26.521517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 919/1853] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:26.526623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9084-9676-451f-a31d-48c44cfc9bcd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.561000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 935/1854] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:26 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:26.561742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9084-9676-451f-a31d-48c44cfc9bcd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:26.563528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1a46d-e6e0-4981-9c66-317bb339a0db admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.567483 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-12f1a46d-e6e0-4981-9c66-317bb339a0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:26.579501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dfbe4-3598-4536-ae61-efc5b93bcddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:26.583040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dfbe4-3598-4536-ae61-efc5b93bcddc admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.585247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dfbe4-3598-4536-ae61-efc5b93bcddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.586600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4dfbe4-3598-4536-ae61-efc5b93bcddc admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.590154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dfbe4-3598-4536-ae61-efc5b93bcddc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:36:26.592535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dfbe4-3598-4536-ae61-efc5b93bcddc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:26.597408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4dfbe4-3598-4536-ae61-efc5b93bcddc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:26.600164 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1a46d-e6e0-4981-9c66-317bb339a0db admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:26.613431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556e7cd-50bc-40a7-90fb-f1ff30941f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:26.614426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556e7cd-50bc-40a7-90fb-f1ff30941f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.614959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556e7cd-50bc-40a7-90fb-f1ff30941f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.615427 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556e7cd-50bc-40a7-90fb-f1ff30941f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.616358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556e7cd-50bc-40a7-90fb-f1ff30941f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:26.618610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556e7cd-50bc-40a7-90fb-f1ff30941f91 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:26.641178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 936/1855] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:26.642937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556e7cd-50bc-40a7-90fb-f1ff30941f91 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.645135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 920/1856] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:36:26.645410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b556e7cd-50bc-40a7-90fb-f1ff30941f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:26.646027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dfbe4-3598-4536-ae61-efc5b93bcddc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.672612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4dfbe4-3598-4536-ae61-efc5b93bcddc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:26.675620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556e7cd-50bc-40a7-90fb-f1ff30941f91 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:26.681170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:26.681389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8040b47-1b2e-4299-a64d-ad5ae32b6acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:26.681989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8040b47-1b2e-4299-a64d-ad5ae32b6acb admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.682815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.683454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.684408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8040b47-1b2e-4299-a64d-ad5ae32b6acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.684638 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] PATH_INFO: `/v3/projects/88650ab507644dbbbce797b7ca4f480f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.684752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8040b47-1b2e-4299-a64d-ad5ae32b6acb admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.685516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8040b47-1b2e-4299-a64d-ad5ae32b6acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:26.687427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8040b47-1b2e-4299-a64d-ad5ae32b6acb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:26.688447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] RBAC: Authorizing `identity:delete_project(project_id=88650ab507644dbbbce797b7ca4f480f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:26.696784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:26.701644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 921/1857] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:26.702578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8040b47-1b2e-4299-a64d-ad5ae32b6acb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.713609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8040b47-1b2e-4299-a64d-ad5ae32b6acb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:26.721405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9409c5-a1b6-4069-90d8-1021954e8bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:26.722155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9409c5-a1b6-4069-90d8-1021954e8bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.722451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9409c5-a1b6-4069-90d8-1021954e8bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.722797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9409c5-a1b6-4069-90d8-1021954e8bd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.723509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9409c5-a1b6-4069-90d8-1021954e8bd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:26.725519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9409c5-a1b6-4069-90d8-1021954e8bd3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:26.771090 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 922/1858] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:26 2022] POST /identity/v3/projects => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:26.772284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9409c5-a1b6-4069-90d8-1021954e8bd3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.810004 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88650ab507644dbbbce797b7ca4f480f', 'request_id': 'req-bad13d94-083d-4b2b-91ae-594aefaf73df'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:26.821400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88650ab507644dbbbce797b7ca4f480f', 'request_id': 'req-bad13d94-083d-4b2b-91ae-594aefaf73df'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:26.832382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:26.840257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88650ab507644dbbbce797b7ca4f480f', 'request_id': 'req-bad13d94-083d-4b2b-91ae-594aefaf73df'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:26.860457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] The token cache is being invalidate because project 88650ab507644dbbbce797b7ca4f480f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:26.860795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:26.861532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:26.863231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 937/1859] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:26 2022] DELETE /identity/v3/projects/88650ab507644dbbbce797b7ca4f480f => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:26.868037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.869393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:26.872097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9409c5-a1b6-4069-90d8-1021954e8bd3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:26.936676 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8572b01-3dd2-4b14-8148-d41f373947ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:26.938806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad13d94-083d-4b2b-91ae-594aefaf73df admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:26.939057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8572b01-3dd2-4b14-8148-d41f373947ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.939478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8572b01-3dd2-4b14-8148-d41f373947ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.939768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8572b01-3dd2-4b14-8148-d41f373947ac admin admin] PATH_INFO: `/v3/projects/ed2483111483446dad226d6bee3f6990/users/e00fcdbafeb8405388c6103c4e0f9eb0/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.944035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dd2a7-f103-4610-af01-e78217e1f4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:26.944721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dd2a7-f103-4610-af01-e78217e1f4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.945028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dd2a7-f103-4610-af01-e78217e1f4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.945281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dd2a7-f103-4610-af01-e78217e1f4c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:26.945977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dd2a7-f103-4610-af01-e78217e1f4c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:26.947945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dd2a7-f103-4610-af01-e78217e1f4c5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:26.958446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8572b01-3dd2-4b14-8148-d41f373947ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2483111483446dad226d6bee3f6990, user_id=e00fcdbafeb8405388c6103c4e0f9eb0, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:26.961498 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8572b01-3dd2-4b14-8148-d41f373947ac admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:26.967277 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:26.978213 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 938/1861] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:26 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:26.978426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 923/1861] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:26 2022] PUT /identity/v3/projects/ed2483111483446dad226d6bee3f6990/users/e00fcdbafeb8405388c6103c4e0f9eb0/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:26.988285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8572b01-3dd2-4b14-8148-d41f373947ac admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:26.988465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc23d7c-c695-49f5-be74-73c27b9d1836 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:26.988465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc23d7c-c695-49f5-be74-73c27b9d1836 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:26.988465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc23d7c-c695-49f5-be74-73c27b9d1836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.009463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4dc23d7c-c695-49f5-be74-73c27b9d1836 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:27.010917 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4dc23d7c-c695-49f5-be74-73c27b9d1836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:27.051560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8572b01-3dd2-4b14-8148-d41f373947ac admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.056447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f9b4b-be32-485d-bf22-c5385e3da007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.057120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f9b4b-be32-485d-bf22-c5385e3da007 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.057390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f9b4b-be32-485d-bf22-c5385e3da007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.057661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f9b4b-be32-485d-bf22-c5385e3da007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.058417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f9b4b-be32-485d-bf22-c5385e3da007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.060046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f9b4b-be32-485d-bf22-c5385e3da007 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.068952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 939/1862] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:27.070117 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 924/1863] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:27.071126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1551986-c91c-4057-88af-d403001627b4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.071577 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1551986-c91c-4057-88af-d403001627b4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.071862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1551986-c91c-4057-88af-d403001627b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.073942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f9b4b-be32-485d-bf22-c5385e3da007 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.083683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f9b4b-be32-485d-bf22-c5385e3da007 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.089003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05eaff8-ecbf-480c-a14d-199c2da55e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.090026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eaff8-ecbf-480c-a14d-199c2da55e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.090333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eaff8-ecbf-480c-a14d-199c2da55e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.090653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05eaff8-ecbf-480c-a14d-199c2da55e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.091412 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05eaff8-ecbf-480c-a14d-199c2da55e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.093018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05eaff8-ecbf-480c-a14d-199c2da55e3f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.099718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f1551986-c91c-4057-88af-d403001627b4 None None] MFA Rules not processed for user `e00fcdbafeb8405388c6103c4e0f9eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:27.100912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f1551986-c91c-4057-88af-d403001627b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:27.103801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 925/1864] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:36:27.104787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64011b1-c83f-469d-be60-483fb76b9d8f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.105077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64011b1-c83f-469d-be60-483fb76b9d8f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.105354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64011b1-c83f-469d-be60-483fb76b9d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.123556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a64011b1-c83f-469d-be60-483fb76b9d8f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:27.125045 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a64011b1-c83f-469d-be60-483fb76b9d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:27.132869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 926/1865] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:27.133619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64011b1-c83f-469d-be60-483fb76b9d8f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.142986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64011b1-c83f-469d-be60-483fb76b9d8f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.148029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0de0dc-710a-43f5-a244-81ecb08849a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.148905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de0dc-710a-43f5-a244-81ecb08849a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.149185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de0dc-710a-43f5-a244-81ecb08849a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.149453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0de0dc-710a-43f5-a244-81ecb08849a6 admin admin] PATH_INFO: `/v3/projects/02b6093b6e8b4d5d9e0b76718c37cf98/users/83510f13ca214815b333fe5fa0c79f71/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.164501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0de0dc-710a-43f5-a244-81ecb08849a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b6093b6e8b4d5d9e0b76718c37cf98, user_id=83510f13ca214815b333fe5fa0c79f71, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.167172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0de0dc-710a-43f5-a244-81ecb08849a6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.171219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 940/1866] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:27.172423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1551986-c91c-4057-88af-d403001627b4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.182373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 927/1867] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:27 2022] PUT /identity/v3/projects/02b6093b6e8b4d5d9e0b76718c37cf98/users/83510f13ca214815b333fe5fa0c79f71/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:27.183210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0de0dc-710a-43f5-a244-81ecb08849a6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.184579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1551986-c91c-4057-88af-d403001627b4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.189221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-da0de0dc-710a-43f5-a244-81ecb08849a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:27.210811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:27.248684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc066c-26bb-49df-9c66-ed67edab894c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.249400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc066c-26bb-49df-9c66-ed67edab894c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.249947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc066c-26bb-49df-9c66-ed67edab894c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.250375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc066c-26bb-49df-9c66-ed67edab894c admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.251160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc066c-26bb-49df-9c66-ed67edab894c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:36:27.251470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc066c-26bb-49df-9c66-ed67edab894c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.253125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc066c-26bb-49df-9c66-ed67edab894c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.257906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0de0dc-710a-43f5-a244-81ecb08849a6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.263051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef3252-9f03-418c-8ad3-8b5a63c0b7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.263726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef3252-9f03-418c-8ad3-8b5a63c0b7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.263998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef3252-9f03-418c-8ad3-8b5a63c0b7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.264277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef3252-9f03-418c-8ad3-8b5a63c0b7b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.264973 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef3252-9f03-418c-8ad3-8b5a63c0b7b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.265486 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 941/1868] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:36:27.266774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df570c-1812-440b-8c4f-0bc941cdcf4c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.267058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df570c-1812-440b-8c4f-0bc941cdcf4c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.267151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef3252-9f03-418c-8ad3-8b5a63c0b7b0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.267331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df570c-1812-440b-8c4f-0bc941cdcf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.297334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-28df570c-1812-440b-8c4f-0bc941cdcf4c None None] MFA Rules not processed for user `83510f13ca214815b333fe5fa0c79f71`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:27.298538 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-28df570c-1812-440b-8c4f-0bc941cdcf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:27.298823 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 928/1869] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:27 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:27.299759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef3252-9f03-418c-8ad3-8b5a63c0b7b0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.320403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:27.414715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 942/1870] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:27.415512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df570c-1812-440b-8c4f-0bc941cdcf4c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.428156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef3252-9f03-418c-8ad3-8b5a63c0b7b0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.432671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df570c-1812-440b-8c4f-0bc941cdcf4c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.439992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c097847-c49e-4be1-ac05-6289ea7e2067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.440687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c097847-c49e-4be1-ac05-6289ea7e2067 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.440973 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c097847-c49e-4be1-ac05-6289ea7e2067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.441252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c097847-c49e-4be1-ac05-6289ea7e2067 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.441992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c097847-c49e-4be1-ac05-6289ea7e2067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.443934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c097847-c49e-4be1-ac05-6289ea7e2067 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.444786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc21de-5b43-403c-a0d5-10b94cc8319b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.445949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc21de-5b43-403c-a0d5-10b94cc8319b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.446570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc21de-5b43-403c-a0d5-10b94cc8319b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.448005 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc21de-5b43-403c-a0d5-10b94cc8319b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.449094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc21de-5b43-403c-a0d5-10b94cc8319b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.451763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc21de-5b43-403c-a0d5-10b94cc8319b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.483555 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 929/1871] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:27 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:27.484619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 943/1872] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:27 2022] POST /identity/v3/users => generated 294 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:27.484891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c097847-c49e-4be1-ac05-6289ea7e2067 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.491272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc21de-5b43-403c-a0d5-10b94cc8319b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.503272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc21de-5b43-403c-a0d5-10b94cc8319b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.564453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c097847-c49e-4be1-ac05-6289ea7e2067 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.569256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4513017-590a-4b6f-ad8f-7ffd2701a852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.569939 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4513017-590a-4b6f-ad8f-7ffd2701a852 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.570475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4513017-590a-4b6f-ad8f-7ffd2701a852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.570475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4513017-590a-4b6f-ad8f-7ffd2701a852 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.571226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4513017-590a-4b6f-ad8f-7ffd2701a852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.573149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4513017-590a-4b6f-ad8f-7ffd2701a852 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.579009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c6701-ff53-40ac-ae42-bdeefed14cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.580439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c6701-ff53-40ac-ae42-bdeefed14cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.580926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c6701-ff53-40ac-ae42-bdeefed14cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.581369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c6701-ff53-40ac-ae42-bdeefed14cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.583382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c6701-ff53-40ac-ae42-bdeefed14cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.586990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c6701-ff53-40ac-ae42-bdeefed14cfe admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.611537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 944/1874] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:36:27.613009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c6701-ff53-40ac-ae42-bdeefed14cfe admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.613628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 930/1873] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:27 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:27.615065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4513017-590a-4b6f-ad8f-7ffd2701a852 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.625441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c6701-ff53-40ac-ae42-bdeefed14cfe admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.636437 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:27.717926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4513017-590a-4b6f-ad8f-7ffd2701a852 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.723306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0b425-ef16-49ea-885f-340c8f8f5391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.723851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b425-ef16-49ea-885f-340c8f8f5391 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.724167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b425-ef16-49ea-885f-340c8f8f5391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.724403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0b425-ef16-49ea-885f-340c8f8f5391 admin admin] PATH_INFO: `/v3/projects/8e24b80d85e54768961b308e7c6bd3ca/users/e96d41a212304674a0c2be0488f8fc0b/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.729486 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72806b76-5a71-4a80-8563-34666352ed0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.730274 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72806b76-5a71-4a80-8563-34666352ed0b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.730560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72806b76-5a71-4a80-8563-34666352ed0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.730863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72806b76-5a71-4a80-8563-34666352ed0b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.732051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72806b76-5a71-4a80-8563-34666352ed0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.734429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72806b76-5a71-4a80-8563-34666352ed0b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.740044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0b425-ef16-49ea-885f-340c8f8f5391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e24b80d85e54768961b308e7c6bd3ca, user_id=e96d41a212304674a0c2be0488f8fc0b, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.742956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0b425-ef16-49ea-885f-340c8f8f5391 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.759649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 945/1875] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:27 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:27.760216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 931/1876] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:27 2022] PUT /identity/v3/projects/8e24b80d85e54768961b308e7c6bd3ca/users/e96d41a212304674a0c2be0488f8fc0b/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:27.760901 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72806b76-5a71-4a80-8563-34666352ed0b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.767046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0b425-ef16-49ea-885f-340c8f8f5391 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.771015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72806b76-5a71-4a80-8563-34666352ed0b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.790801 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:27.835224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267242a2-4cac-473d-8997-b0a15890a580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.836370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267242a2-4cac-473d-8997-b0a15890a580 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.836891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267242a2-4cac-473d-8997-b0a15890a580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.837246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267242a2-4cac-473d-8997-b0a15890a580 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.838024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267242a2-4cac-473d-8997-b0a15890a580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.838992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0b425-ef16-49ea-885f-340c8f8f5391 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.840505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267242a2-4cac-473d-8997-b0a15890a580 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.844187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928aab1-cf4f-483a-bd67-324a50ec33e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.844868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928aab1-cf4f-483a-bd67-324a50ec33e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.845152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928aab1-cf4f-483a-bd67-324a50ec33e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.845426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c928aab1-cf4f-483a-bd67-324a50ec33e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.846158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928aab1-cf4f-483a-bd67-324a50ec33e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.847803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c928aab1-cf4f-483a-bd67-324a50ec33e1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.860129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 932/1877] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:36:27.861014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928aab1-cf4f-483a-bd67-324a50ec33e1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.875966 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c928aab1-cf4f-483a-bd67-324a50ec33e1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.876115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 946/1878] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:27 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:27.877215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267242a2-4cac-473d-8997-b0a15890a580 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.884140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c164f5-268f-4829-842f-76d9802bd2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.884835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c164f5-268f-4829-842f-76d9802bd2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.885119 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c164f5-268f-4829-842f-76d9802bd2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.885442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c164f5-268f-4829-842f-76d9802bd2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.886230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c164f5-268f-4829-842f-76d9802bd2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.888234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c164f5-268f-4829-842f-76d9802bd2b0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.893619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267242a2-4cac-473d-8997-b0a15890a580 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.901353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbb7a0-c027-43cb-b55b-77f9a77dbd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.902298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbb7a0-c027-43cb-b55b-77f9a77dbd46 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.903065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbb7a0-c027-43cb-b55b-77f9a77dbd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.903509 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbb7a0-c027-43cb-b55b-77f9a77dbd46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.905581 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 933/1879] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:27.906655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c164f5-268f-4829-842f-76d9802bd2b0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.911134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbb7a0-c027-43cb-b55b-77f9a77dbd46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.912949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbb7a0-c027-43cb-b55b-77f9a77dbd46 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.917583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c164f5-268f-4829-842f-76d9802bd2b0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.923044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357cbbb-32e4-45df-ae96-0ed714f3afd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.923759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357cbbb-32e4-45df-ae96-0ed714f3afd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.924114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357cbbb-32e4-45df-ae96-0ed714f3afd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.924462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357cbbb-32e4-45df-ae96-0ed714f3afd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.925274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357cbbb-32e4-45df-ae96-0ed714f3afd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.926856 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 947/1880] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:27.927002 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357cbbb-32e4-45df-ae96-0ed714f3afd5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.928089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbb7a0-c027-43cb-b55b-77f9a77dbd46 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.941936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 934/1881] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:27.942112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbb7a0-c027-43cb-b55b-77f9a77dbd46 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.942191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357cbbb-32e4-45df-ae96-0ed714f3afd5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.945034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dc907-c74f-410b-a575-38fbd3c1029d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.945634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dc907-c74f-410b-a575-38fbd3c1029d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.945918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dc907-c74f-410b-a575-38fbd3c1029d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.946191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dc907-c74f-410b-a575-38fbd3c1029d admin admin] PATH_INFO: `/v3/projects/8e24b80d85e54768961b308e7c6bd3ca/users/e96d41a212304674a0c2be0488f8fc0b/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.948224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dc907-c74f-410b-a575-38fbd3c1029d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e24b80d85e54768961b308e7c6bd3ca, user_id=e96d41a212304674a0c2be0488f8fc0b, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.950902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dc907-c74f-410b-a575-38fbd3c1029d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:27.951494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357cbbb-32e4-45df-ae96-0ed714f3afd5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:27.956945 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82eb3c-3471-40c4-b466-80e7f58ac9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:27.957880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82eb3c-3471-40c4-b466-80e7f58ac9c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:27.958167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82eb3c-3471-40c4-b466-80e7f58ac9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:27.958450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82eb3c-3471-40c4-b466-80e7f58ac9c9 admin admin] PATH_INFO: `/v3/projects/d9947482551847a7887759ff57adf0e0/users/d79a503344bc41e1a027b953e27421d6/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:27.964153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 948/1882] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:27 2022] PUT /identity/v3/projects/8e24b80d85e54768961b308e7c6bd3ca/users/e96d41a212304674a0c2be0488f8fc0b/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:27.965086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dc907-c74f-410b-a575-38fbd3c1029d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:27.987005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82eb3c-3471-40c4-b466-80e7f58ac9c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9947482551847a7887759ff57adf0e0, user_id=d79a503344bc41e1a027b953e27421d6, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:27.989736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82eb3c-3471-40c4-b466-80e7f58ac9c9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:28.014884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 935/1883] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:27 2022] PUT /identity/v3/projects/d9947482551847a7887759ff57adf0e0/users/d79a503344bc41e1a027b953e27421d6/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:28.014884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd94616-5c5e-4b56-a25f-bc8351974c08 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.014884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd94616-5c5e-4b56-a25f-bc8351974c08 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.014884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd94616-5c5e-4b56-a25f-bc8351974c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.054512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4fd94616-5c5e-4b56-a25f-bc8351974c08 None None] MFA Rules not processed for user `e96d41a212304674a0c2be0488f8fc0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:28.055771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4fd94616-5c5e-4b56-a25f-bc8351974c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:28.060060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dc907-c74f-410b-a575-38fbd3c1029d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:28.098596 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:28.120527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbeb63-16a9-44c8-8154-3f52a78b48a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:28.121490 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbeb63-16a9-44c8-8154-3f52a78b48a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.121829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbeb63-16a9-44c8-8154-3f52a78b48a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.122112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbeb63-16a9-44c8-8154-3f52a78b48a0 admin admin] PATH_INFO: `/v3/projects/71b7c308e6ab481b85ca2909e5105248/users/62e66bc2a6cf4025afb3aa9a87eab650/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.131018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 936/1884] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:28.132388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2015b-2f66-42eb-918e-828eb8db0bf4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.132686 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2015b-2f66-42eb-918e-828eb8db0bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.133024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2015b-2f66-42eb-918e-828eb8db0bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.137967 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbeb63-16a9-44c8-8154-3f52a78b48a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b7c308e6ab481b85ca2909e5105248, user_id=62e66bc2a6cf4025afb3aa9a87eab650, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:28.140853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbeb63-16a9-44c8-8154-3f52a78b48a0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:28.154968 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 949/1885] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:27 2022] PUT /identity/v3/projects/71b7c308e6ab481b85ca2909e5105248/users/62e66bc2a6cf4025afb3aa9a87eab650/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:28.155769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbeb63-16a9-44c8-8154-3f52a78b48a0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:28.157386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-22fbeb63-16a9-44c8-8154-3f52a78b48a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:28.167213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c0e2015b-2f66-42eb-918e-828eb8db0bf4 None None] MFA Rules not processed for user `d79a503344bc41e1a027b953e27421d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:28.168379 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c0e2015b-2f66-42eb-918e-828eb8db0bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:28.179430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:28.227747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbeb63-16a9-44c8-8154-3f52a78b48a0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:28.236998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b0476-4aff-4847-af7a-1f99b157182e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:28.238215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0476-4aff-4847-af7a-1f99b157182e service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.238508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0476-4aff-4847-af7a-1f99b157182e service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.239298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b0476-4aff-4847-af7a-1f99b157182e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.272840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 937/1886] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:28.273719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2015b-2f66-42eb-918e-828eb8db0bf4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:28.341776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b0476-4aff-4847-af7a-1f99b157182e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:28.343353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8b0476-4aff-4847-af7a-1f99b157182e service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:28.353129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 950/1887] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:28.354038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b0476-4aff-4847-af7a-1f99b157182e service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:28.355444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3e8b0476-4aff-4847-af7a-1f99b157182e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:28.368453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b0476-4aff-4847-af7a-1f99b157182e service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:28.372689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2015b-2f66-42eb-918e-828eb8db0bf4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:28.373839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88ee08-8010-43c1-9645-bcb02ffd2251 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:28.374514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88ee08-8010-43c1-9645-bcb02ffd2251 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.375019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88ee08-8010-43c1-9645-bcb02ffd2251 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.375414 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e88ee08-8010-43c1-9645-bcb02ffd2251 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.381280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b03aa-f36a-4e6f-8a43-0a761c3b1ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:28.381971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b03aa-f36a-4e6f-8a43-0a761c3b1ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.382261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b03aa-f36a-4e6f-8a43-0a761c3b1ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.382546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b03aa-f36a-4e6f-8a43-0a761c3b1ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.383355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b03aa-f36a-4e6f-8a43-0a761c3b1ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:28.384985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88ee08-8010-43c1-9645-bcb02ffd2251 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:28.385111 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b03aa-f36a-4e6f-8a43-0a761c3b1ac3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:28.386560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e88ee08-8010-43c1-9645-bcb02ffd2251 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:28.396621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 951/1888] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:28.400282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 938/1889] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:28.403795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88ee08-8010-43c1-9645-bcb02ffd2251 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:28.423939 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b03aa-f36a-4e6f-8a43-0a761c3b1ac3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:28.424150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e88ee08-8010-43c1-9645-bcb02ffd2251 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:28.428931 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d683a-1478-44db-bc4e-3369452dddb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:28.429910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d683a-1478-44db-bc4e-3369452dddb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.430136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d683a-1478-44db-bc4e-3369452dddb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.430409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d683a-1478-44db-bc4e-3369452dddb2 admin admin] PATH_INFO: `/v3/projects/71b7c308e6ab481b85ca2909e5105248/users/62e66bc2a6cf4025afb3aa9a87eab650/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.432437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d683a-1478-44db-bc4e-3369452dddb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b7c308e6ab481b85ca2909e5105248, user_id=62e66bc2a6cf4025afb3aa9a87eab650, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:28.435175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d683a-1478-44db-bc4e-3369452dddb2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:28.444824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b03aa-f36a-4e6f-8a43-0a761c3b1ac3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:28.446773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 952/1890] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:28 2022] PUT /identity/v3/projects/71b7c308e6ab481b85ca2909e5105248/users/62e66bc2a6cf4025afb3aa9a87eab650/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:28.459321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bd724-4795-4b54-8612-f84e6bdc1e85 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.459821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bd724-4795-4b54-8612-f84e6bdc1e85 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.460864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bd724-4795-4b54-8612-f84e6bdc1e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.504611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0d6bd724-4795-4b54-8612-f84e6bdc1e85 None None] MFA Rules not processed for user `62e66bc2a6cf4025afb3aa9a87eab650`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:28.506933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0d6bd724-4795-4b54-8612-f84e6bdc1e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:28.541293 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fac9c-d886-489b-9d6f-e18638dec9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:28.542183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fac9c-d886-489b-9d6f-e18638dec9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.542463 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fac9c-d886-489b-9d6f-e18638dec9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.542922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fac9c-d886-489b-9d6f-e18638dec9dc admin admin] PATH_INFO: `/v3/users/d79a503344bc41e1a027b953e27421d6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.544745 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fac9c-d886-489b-9d6f-e18638dec9dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d79a503344bc41e1a027b953e27421d6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:28.548042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fac9c-d886-489b-9d6f-e18638dec9dc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:28.550645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:28.608842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 953/1891] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:28 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:28.620735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a41fac9c-d886-489b-9d6f-e18638dec9dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd79a503344bc41e1a027b953e27421d6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:28.620922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bd724-4795-4b54-8612-f84e6bdc1e85 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:28.622308 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a41fac9c-d886-489b-9d6f-e18638dec9dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd79a503344bc41e1a027b953e27421d6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:28.637879 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a41fac9c-d886-489b-9d6f-e18638dec9dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd79a503344bc41e1a027b953e27421d6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:28.638762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a41fac9c-d886-489b-9d6f-e18638dec9dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd79a503344bc41e1a027b953e27421d6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:28.647217 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a41fac9c-d886-489b-9d6f-e18638dec9dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd79a503344bc41e1a027b953e27421d6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:28.660804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:28.680099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 939/1892] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:28 2022] DELETE /identity/v3/users/d79a503344bc41e1a027b953e27421d6 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:28.685080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd87477-79c2-4bf4-bf10-52b1b45ba36e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.685875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd87477-79c2-4bf4-bf10-52b1b45ba36e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.686300 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd87477-79c2-4bf4-bf10-52b1b45ba36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.710587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9cd87477-79c2-4bf4-bf10-52b1b45ba36e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:28.712413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9cd87477-79c2-4bf4-bf10-52b1b45ba36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:28.739398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bd724-4795-4b54-8612-f84e6bdc1e85 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:28.804017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 940/1893] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:28.826209 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd87477-79c2-4bf4-bf10-52b1b45ba36e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:28.829510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9cd87477-79c2-4bf4-bf10-52b1b45ba36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:28.877478 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f240b13-df7c-4d13-af23-31846af50bba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:28.878206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f240b13-df7c-4d13-af23-31846af50bba service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.878441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f240b13-df7c-4d13-af23-31846af50bba service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.878737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f240b13-df7c-4d13-af23-31846af50bba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.886971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2f240b13-df7c-4d13-af23-31846af50bba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:28.914229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd87477-79c2-4bf4-bf10-52b1b45ba36e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:28.918185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a3b32-d4ca-43a9-a607-aab948d7982f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:28.919195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a3b32-d4ca-43a9-a607-aab948d7982f service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:28.919861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a3b32-d4ca-43a9-a607-aab948d7982f service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:28.920955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a3b32-d4ca-43a9-a607-aab948d7982f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:28.935685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a3b32-d4ca-43a9-a607-aab948d7982f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:28.938980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a3b32-d4ca-43a9-a607-aab948d7982f service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:28.952237 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 941/1894] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:29.004862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f240b13-df7c-4d13-af23-31846af50bba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:29.007062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f240b13-df7c-4d13-af23-31846af50bba service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:29.021874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 954/1895] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:28 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:29.078138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a3b32-d4ca-43a9-a607-aab948d7982f service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:29.079991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-825a3b32-d4ca-43a9-a607-aab948d7982f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:29.097281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a3b32-d4ca-43a9-a607-aab948d7982f service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:29.102169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216c623-ebb0-4c60-82c9-89f2cd7f320d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:29.103090 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216c623-ebb0-4c60-82c9-89f2cd7f320d service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:29.103558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216c623-ebb0-4c60-82c9-89f2cd7f320d service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:29.104042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216c623-ebb0-4c60-82c9-89f2cd7f320d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:29.106247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1216c623-ebb0-4c60-82c9-89f2cd7f320d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:29.129420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:29.186334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216c623-ebb0-4c60-82c9-89f2cd7f320d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:29.187882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216c623-ebb0-4c60-82c9-89f2cd7f320d service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:29.199644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 942/1896] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:29.308145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f240b13-df7c-4d13-af23-31846af50bba service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:29.309624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2f240b13-df7c-4d13-af23-31846af50bba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:29.326591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f240b13-df7c-4d13-af23-31846af50bba service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:29.334785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:29.335979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:29.336264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:29.336540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] PATH_INFO: `/v3/projects/d9947482551847a7887759ff57adf0e0` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:29.337777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9947482551847a7887759ff57adf0e0)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:29.340112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:29.392900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9947482551847a7887759ff57adf0e0', 'request_id': 'req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:29.399248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9947482551847a7887759ff57adf0e0', 'request_id': 'req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:29.405003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9947482551847a7887759ff57adf0e0', 'request_id': 'req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:29.419942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] The token cache is being invalidate because project d9947482551847a7887759ff57adf0e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:29.420227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:29.420841 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e827d03b-ddf1-4758-8fb7-a40cc0a61a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:29.422454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 955/1897] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:29 2022] DELETE /identity/v3/projects/d9947482551847a7887759ff57adf0e0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:29.427648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b958ea1-6162-4c81-b63f-43cf28b812d4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:29.428017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b958ea1-6162-4c81-b63f-43cf28b812d4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:29.428358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b958ea1-6162-4c81-b63f-43cf28b812d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:29.447710 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9b958ea1-6162-4c81-b63f-43cf28b812d4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:29.448843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9b958ea1-6162-4c81-b63f-43cf28b812d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:29.539049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 943/1898] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:36:29.540998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a80a70-97cd-4675-9685-96efeff074ce None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:29.541298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a80a70-97cd-4675-9685-96efeff074ce None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:29.541602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a80a70-97cd-4675-9685-96efeff074ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:29.579751 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-41a80a70-97cd-4675-9685-96efeff074ce None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:29.581051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-41a80a70-97cd-4675-9685-96efeff074ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:29.589624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 956/1899] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:29.594554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b958ea1-6162-4c81-b63f-43cf28b812d4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:29.605082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b958ea1-6162-4c81-b63f-43cf28b812d4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:29.611777 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3d2e2-dc11-4446-b830-87679ffe0c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:29.612552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3d2e2-dc11-4446-b830-87679ffe0c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:29.612841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3d2e2-dc11-4446-b830-87679ffe0c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:29.613121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3d2e2-dc11-4446-b830-87679ffe0c17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:29.613869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3d2e2-dc11-4446-b830-87679ffe0c17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:36:29.614182 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3d2e2-dc11-4446-b830-87679ffe0c17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:29.615829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3d2e2-dc11-4446-b830-87679ffe0c17 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:29.631844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 944/1900] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:36:29.632950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a80a70-97cd-4675-9685-96efeff074ce None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:29.642370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a80a70-97cd-4675-9685-96efeff074ce None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:29.647002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14eaf38-5909-4de5-a457-acae86cfc095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:29.647667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eaf38-5909-4de5-a457-acae86cfc095 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:29.647956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eaf38-5909-4de5-a457-acae86cfc095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:29.648241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14eaf38-5909-4de5-a457-acae86cfc095 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:29.648949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14eaf38-5909-4de5-a457-acae86cfc095 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:29.650892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14eaf38-5909-4de5-a457-acae86cfc095 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:29.661365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:29.686503 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 957/1901] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:29 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:36:29.688473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3d2e2-dc11-4446-b830-87679ffe0c17 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:29.700039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3d2e2-dc11-4446-b830-87679ffe0c17 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:29.766799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9bce5-deef-4003-ae49-5dee4c8462d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:29.767989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9bce5-deef-4003-ae49-5dee4c8462d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:29.768486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9bce5-deef-4003-ae49-5dee4c8462d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:29.768718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9bce5-deef-4003-ae49-5dee4c8462d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:29.769438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9bce5-deef-4003-ae49-5dee4c8462d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:29.771768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9bce5-deef-4003-ae49-5dee4c8462d3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:29.795342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 945/1902] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:29 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:36:29.799603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14eaf38-5909-4de5-a457-acae86cfc095 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:29.810163 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14eaf38-5909-4de5-a457-acae86cfc095 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:29.815019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4557b3-abd8-479c-be01-30cf0a8f7a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:29.815689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4557b3-abd8-479c-be01-30cf0a8f7a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:29.815971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4557b3-abd8-479c-be01-30cf0a8f7a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:29.816247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4557b3-abd8-479c-be01-30cf0a8f7a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:29.816984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4557b3-abd8-479c-be01-30cf0a8f7a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:29.818590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4557b3-abd8-479c-be01-30cf0a8f7a2a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:29.829713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 958/1903] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:29.833738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9bce5-deef-4003-ae49-5dee4c8462d3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:29.844266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9bce5-deef-4003-ae49-5dee4c8462d3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:29.848871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2565d-ba02-4043-9f39-44d832342f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:29.849535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2565d-ba02-4043-9f39-44d832342f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:29.849820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2565d-ba02-4043-9f39-44d832342f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:29.850100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2565d-ba02-4043-9f39-44d832342f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:29.850861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2565d-ba02-4043-9f39-44d832342f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:29.852465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2565d-ba02-4043-9f39-44d832342f7b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:29.869442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 946/1904] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:29.875247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4557b3-abd8-479c-be01-30cf0a8f7a2a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:29.887684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4557b3-abd8-479c-be01-30cf0a8f7a2a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:29.894039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c6bfc-4039-4278-84b5-d064cd76e737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:29.894966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c6bfc-4039-4278-84b5-d064cd76e737 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:29.895241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c6bfc-4039-4278-84b5-d064cd76e737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:29.895512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c6bfc-4039-4278-84b5-d064cd76e737 admin admin] PATH_INFO: `/v3/projects/02f8382872514195b5d03b46775f7485/users/3958ea727248421ca8277aaa29fa10e4/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:29.911871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c6bfc-4039-4278-84b5-d064cd76e737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02f8382872514195b5d03b46775f7485, user_id=3958ea727248421ca8277aaa29fa10e4, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:29.914673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c6bfc-4039-4278-84b5-d064cd76e737 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:29.931085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 959/1905] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:29 2022] PUT /identity/v3/projects/02f8382872514195b5d03b46775f7485/users/3958ea727248421ca8277aaa29fa10e4/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:36:29.942750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630bd11-6dae-4fce-bc2f-f7e781a0190c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:29.943079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630bd11-6dae-4fce-bc2f-f7e781a0190c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:29.943358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630bd11-6dae-4fce-bc2f-f7e781a0190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:29.971659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a630bd11-6dae-4fce-bc2f-f7e781a0190c None None] MFA Rules not processed for user `3958ea727248421ca8277aaa29fa10e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:29.972782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a630bd11-6dae-4fce-bc2f-f7e781a0190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:29.992095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:30.045846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 947/1906] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:30.056988 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c6bfc-4039-4278-84b5-d064cd76e737 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:30.058826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f32c6bfc-4039-4278-84b5-d064cd76e737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:30.133820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c6bfc-4039-4278-84b5-d064cd76e737 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:30.137778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9655f-d85b-4911-8e97-d8e18018b397 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:30.138452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9655f-d85b-4911-8e97-d8e18018b397 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:30.138758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9655f-d85b-4911-8e97-d8e18018b397 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:30.139032 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9655f-d85b-4911-8e97-d8e18018b397 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:30.148521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9655f-d85b-4911-8e97-d8e18018b397 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:30.150014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9655f-d85b-4911-8e97-d8e18018b397 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:30.160215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 960/1907] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:32.600098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630bd11-6dae-4fce-bc2f-f7e781a0190c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:32.602340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a630bd11-6dae-4fce-bc2f-f7e781a0190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:32.677213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630bd11-6dae-4fce-bc2f-f7e781a0190c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:32.683686 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:32.684394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:32.684678 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:32.684953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] PATH_INFO: `/v3/users/83510f13ca214815b333fe5fa0c79f71` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:32.686194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83510f13ca214815b333fe5fa0c79f71)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:32.688339 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:32.736705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83510f13ca214815b333fe5fa0c79f71'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:32.737366 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83510f13ca214815b333fe5fa0c79f71'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:32.749135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83510f13ca214815b333fe5fa0c79f71'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:32.750169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83510f13ca214815b333fe5fa0c79f71'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:32.759013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83510f13ca214815b333fe5fa0c79f71'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:32.772340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 948/1908] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:32 2022] DELETE /identity/v3/users/83510f13ca214815b333fe5fa0c79f71 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:32.777321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9800f-5cab-483f-bffd-16fb7ef05560 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:32.777851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9800f-5cab-483f-bffd-16fb7ef05560 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:32.777851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9800f-5cab-483f-bffd-16fb7ef05560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:32.801458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-22a9800f-5cab-483f-bffd-16fb7ef05560 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:32.802727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-22a9800f-5cab-483f-bffd-16fb7ef05560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:32.881074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 961/1909] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:32.891610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:32.893049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:32.935415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:32.957203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f555c6-deb8-4589-b233-b8324adbfbb3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:32.960981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ba9f2-c59c-42c6-8531-c6ff28b5bda2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:32.961647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ba9f2-c59c-42c6-8531-c6ff28b5bda2 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:32.961937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ba9f2-c59c-42c6-8531-c6ff28b5bda2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:32.962215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ba9f2-c59c-42c6-8531-c6ff28b5bda2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:32.973192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ba9f2-c59c-42c6-8531-c6ff28b5bda2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:32.974646 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548ba9f2-c59c-42c6-8531-c6ff28b5bda2 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:32.985034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 949/1910] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:33.272498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9800f-5cab-483f-bffd-16fb7ef05560 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:33.274082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-22a9800f-5cab-483f-bffd-16fb7ef05560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:33.291154 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9800f-5cab-483f-bffd-16fb7ef05560 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:33.296160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:33.297215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:33.297583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:33.297865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] PATH_INFO: `/v3/projects/02b6093b6e8b4d5d9e0b76718c37cf98` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:33.299102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02b6093b6e8b4d5d9e0b76718c37cf98)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:33.301038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:33.348863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02b6093b6e8b4d5d9e0b76718c37cf98', 'request_id': 'req-68d3bcaa-2106-40c5-afab-029efbb6d627'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:33.355012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02b6093b6e8b4d5d9e0b76718c37cf98', 'request_id': 'req-68d3bcaa-2106-40c5-afab-029efbb6d627'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:33.356236 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:33.361869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02b6093b6e8b4d5d9e0b76718c37cf98', 'request_id': 'req-68d3bcaa-2106-40c5-afab-029efbb6d627'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:33.377316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] The token cache is being invalidate because project 02b6093b6e8b4d5d9e0b76718c37cf98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:33.377603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:33.378241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:33.379917 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 962/1911] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:33 2022] DELETE /identity/v3/projects/02b6093b6e8b4d5d9e0b76718c37cf98 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:33.386066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ba9f2-c59c-42c6-8531-c6ff28b5bda2 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:33.401790 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548ba9f2-c59c-42c6-8531-c6ff28b5bda2 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:33.461311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280f11a-bca6-4d64-acf1-b1700f540349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:33.462191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280f11a-bca6-4d64-acf1-b1700f540349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:33.462503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280f11a-bca6-4d64-acf1-b1700f540349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:33.462807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280f11a-bca6-4d64-acf1-b1700f540349 admin admin] PATH_INFO: `/v3/users/62e66bc2a6cf4025afb3aa9a87eab650` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:33.464143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280f11a-bca6-4d64-acf1-b1700f540349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e66bc2a6cf4025afb3aa9a87eab650)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:33.466309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280f11a-bca6-4d64-acf1-b1700f540349 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:33.518370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a280f11a-bca6-4d64-acf1-b1700f540349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e66bc2a6cf4025afb3aa9a87eab650'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:33.519042 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a280f11a-bca6-4d64-acf1-b1700f540349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e66bc2a6cf4025afb3aa9a87eab650'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:33.531506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a280f11a-bca6-4d64-acf1-b1700f540349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e66bc2a6cf4025afb3aa9a87eab650'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:33.532197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a280f11a-bca6-4d64-acf1-b1700f540349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e66bc2a6cf4025afb3aa9a87eab650'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:33.541691 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a280f11a-bca6-4d64-acf1-b1700f540349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e66bc2a6cf4025afb3aa9a87eab650'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:33.553666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 950/1912] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:33 2022] DELETE /identity/v3/users/62e66bc2a6cf4025afb3aa9a87eab650 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:33.845401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:33.848336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:33.870282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3bcaa-2106-40c5-afab-029efbb6d627 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:33.948134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:33.949242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:33.949528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:33.949804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] PATH_INFO: `/v3/projects/71b7c308e6ab481b85ca2909e5105248` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:33.951039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b7c308e6ab481b85ca2909e5105248)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:33.953353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:34.002631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b7c308e6ab481b85ca2909e5105248', 'request_id': 'req-b1fbb538-d5a6-4c00-acb0-1328a6b71808'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:34.009368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b7c308e6ab481b85ca2909e5105248', 'request_id': 'req-b1fbb538-d5a6-4c00-acb0-1328a6b71808'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:34.016171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b7c308e6ab481b85ca2909e5105248', 'request_id': 'req-b1fbb538-d5a6-4c00-acb0-1328a6b71808'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:34.031719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] The token cache is being invalidate because project 71b7c308e6ab481b85ca2909e5105248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:34.032013 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:34.032664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b1fbb538-d5a6-4c00-acb0-1328a6b71808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:34.034257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 963/1913] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:33 2022] DELETE /identity/v3/projects/71b7c308e6ab481b85ca2909e5105248 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:34.039475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862b3db-96f0-4769-8839-16bea6b17b51 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.039967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862b3db-96f0-4769-8839-16bea6b17b51 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.040268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862b3db-96f0-4769-8839-16bea6b17b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.065718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2862b3db-96f0-4769-8839-16bea6b17b51 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:34.067020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2862b3db-96f0-4769-8839-16bea6b17b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:34.134528 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:34.149644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 951/1914] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:34.161448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160342c8-94d1-41ac-af10-4f7d7244a415 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.161448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160342c8-94d1-41ac-af10-4f7d7244a415 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.161448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160342c8-94d1-41ac-af10-4f7d7244a415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.176242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-160342c8-94d1-41ac-af10-4f7d7244a415 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:34.177442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-160342c8-94d1-41ac-af10-4f7d7244a415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:34.185548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 964/1915] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:34.189959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862b3db-96f0-4769-8839-16bea6b17b51 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:34.199919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862b3db-96f0-4769-8839-16bea6b17b51 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:34.204716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa061e-289f-4a23-946b-4d4a1bfa4546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:34.205389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa061e-289f-4a23-946b-4d4a1bfa4546 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.205756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa061e-289f-4a23-946b-4d4a1bfa4546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.205939 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa061e-289f-4a23-946b-4d4a1bfa4546 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.206681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa061e-289f-4a23-946b-4d4a1bfa4546 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:36:34.207013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa061e-289f-4a23-946b-4d4a1bfa4546 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:34.208613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa061e-289f-4a23-946b-4d4a1bfa4546 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:34.220837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 952/1916] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:36:34.225667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160342c8-94d1-41ac-af10-4f7d7244a415 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:34.236260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160342c8-94d1-41ac-af10-4f7d7244a415 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:34.242387 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922367d6-72cd-4aa1-915f-a776fb2ca90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:34.243107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922367d6-72cd-4aa1-915f-a776fb2ca90c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.243392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922367d6-72cd-4aa1-915f-a776fb2ca90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.243674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922367d6-72cd-4aa1-915f-a776fb2ca90c admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.244379 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922367d6-72cd-4aa1-915f-a776fb2ca90c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:34.246302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922367d6-72cd-4aa1-915f-a776fb2ca90c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:34.275668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 965/1917] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:34 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:34.279498 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa061e-289f-4a23-946b-4d4a1bfa4546 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:34.289793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa061e-289f-4a23-946b-4d4a1bfa4546 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:34.348318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506898a-6017-4204-9dca-ba24e9e9bdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:34.348942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506898a-6017-4204-9dca-ba24e9e9bdca admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.349222 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506898a-6017-4204-9dca-ba24e9e9bdca admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.349501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506898a-6017-4204-9dca-ba24e9e9bdca admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.350196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506898a-6017-4204-9dca-ba24e9e9bdca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:34.352157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506898a-6017-4204-9dca-ba24e9e9bdca admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:34.374635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 953/1918] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:34 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:34.379556 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922367d6-72cd-4aa1-915f-a776fb2ca90c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:34.393736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922367d6-72cd-4aa1-915f-a776fb2ca90c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:34.398428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671617d3-febe-4032-9f8c-4df7cdfe6dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:34.399122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671617d3-febe-4032-9f8c-4df7cdfe6dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.399402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671617d3-febe-4032-9f8c-4df7cdfe6dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.399672 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671617d3-febe-4032-9f8c-4df7cdfe6dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.400428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671617d3-febe-4032-9f8c-4df7cdfe6dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:34.402034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671617d3-febe-4032-9f8c-4df7cdfe6dfe admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:34.412489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 966/1919] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:34.416616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506898a-6017-4204-9dca-ba24e9e9bdca admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:34.427236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506898a-6017-4204-9dca-ba24e9e9bdca admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:34.432599 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcf054-e87e-4856-b5c2-9b5e0b82a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:34.433262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcf054-e87e-4856-b5c2-9b5e0b82a8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.433546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcf054-e87e-4856-b5c2-9b5e0b82a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.433814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcf054-e87e-4856-b5c2-9b5e0b82a8f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.434585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcf054-e87e-4856-b5c2-9b5e0b82a8f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:34.436216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcf054-e87e-4856-b5c2-9b5e0b82a8f2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:34.446704 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 954/1920] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:34.450862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671617d3-febe-4032-9f8c-4df7cdfe6dfe admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:34.461689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671617d3-febe-4032-9f8c-4df7cdfe6dfe admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:34.467999 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f700f-8004-425b-9cc2-4d267daf8441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:34.468906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f700f-8004-425b-9cc2-4d267daf8441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.469186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f700f-8004-425b-9cc2-4d267daf8441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.469457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f700f-8004-425b-9cc2-4d267daf8441 admin admin] PATH_INFO: `/v3/projects/99552574d6b44a3d806c66187c0ef46c/users/255444d4fa644e6192931c97c535b71c/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.485026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f700f-8004-425b-9cc2-4d267daf8441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99552574d6b44a3d806c66187c0ef46c, user_id=255444d4fa644e6192931c97c535b71c, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:34.487696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f700f-8004-425b-9cc2-4d267daf8441 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:34.503961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 967/1921] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:34 2022] PUT /identity/v3/projects/99552574d6b44a3d806c66187c0ef46c/users/255444d4fa644e6192931c97c535b71c/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:34.514751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061934c-460e-49e2-b6d3-84752383c222 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.515045 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061934c-460e-49e2-b6d3-84752383c222 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.515368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061934c-460e-49e2-b6d3-84752383c222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.543078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f061934c-460e-49e2-b6d3-84752383c222 None None] MFA Rules not processed for user `255444d4fa644e6192931c97c535b71c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:34.544297 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f061934c-460e-49e2-b6d3-84752383c222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:34.625335 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 955/1922] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:34.630452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f700f-8004-425b-9cc2-4d267daf8441 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:34.646378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:34.702252 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f700f-8004-425b-9cc2-4d267daf8441 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:34.707594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff471a-4553-482a-9f55-103eedbac593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:34.708267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff471a-4553-482a-9f55-103eedbac593 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.708549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff471a-4553-482a-9f55-103eedbac593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.708822 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff471a-4553-482a-9f55-103eedbac593 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.709564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff471a-4553-482a-9f55-103eedbac593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:34.711523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff471a-4553-482a-9f55-103eedbac593 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:34.743253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 968/1923] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:34 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:34.747872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061934c-460e-49e2-b6d3-84752383c222 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:34.759694 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061934c-460e-49e2-b6d3-84752383c222 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:34.813149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:34.822969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb437d7-10d2-415b-bfa1-4be8a840dfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:34.823648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb437d7-10d2-415b-bfa1-4be8a840dfca admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.823930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb437d7-10d2-415b-bfa1-4be8a840dfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.824204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb437d7-10d2-415b-bfa1-4be8a840dfca admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.824894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb437d7-10d2-415b-bfa1-4be8a840dfca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:34.826849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb437d7-10d2-415b-bfa1-4be8a840dfca admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:34.848409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 956/1924] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:34 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:34.852498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff471a-4553-482a-9f55-103eedbac593 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:34.863106 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff471a-4553-482a-9f55-103eedbac593 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:34.868728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc93b8-c72a-4e59-bce7-24d5e51aec12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:34.869400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc93b8-c72a-4e59-bce7-24d5e51aec12 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.869657 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc93b8-c72a-4e59-bce7-24d5e51aec12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.869932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc93b8-c72a-4e59-bce7-24d5e51aec12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.870665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc93b8-c72a-4e59-bce7-24d5e51aec12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:34.872317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fc93b8-c72a-4e59-bce7-24d5e51aec12 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:34.883993 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 969/1925] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:34.888361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb437d7-10d2-415b-bfa1-4be8a840dfca admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:34.898602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb437d7-10d2-415b-bfa1-4be8a840dfca admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:34.903663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25349ea0-ca99-47c3-af67-613ad452347f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:34.904546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25349ea0-ca99-47c3-af67-613ad452347f admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:34.904826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25349ea0-ca99-47c3-af67-613ad452347f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:34.905160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25349ea0-ca99-47c3-af67-613ad452347f admin admin] PATH_INFO: `/v3/projects/100b8222e80d43bfa02a27793fa4d859/users/66289f3841e4426b89d8580b7a2ee113/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:34.919817 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25349ea0-ca99-47c3-af67-613ad452347f admin admin] RBAC: Authorizing `identity:create_grant(project_id=100b8222e80d43bfa02a27793fa4d859, user_id=66289f3841e4426b89d8580b7a2ee113, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:34.922652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25349ea0-ca99-47c3-af67-613ad452347f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:34.938381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 957/1926] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:34 2022] PUT /identity/v3/projects/100b8222e80d43bfa02a27793fa4d859/users/66289f3841e4426b89d8580b7a2ee113/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:34.942368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc93b8-c72a-4e59-bce7-24d5e51aec12 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:34.954068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc93b8-c72a-4e59-bce7-24d5e51aec12 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:35.013172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fcd03-95c7-4426-9921-8d0e626cba52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:35.013829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fcd03-95c7-4426-9921-8d0e626cba52 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:35.014108 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fcd03-95c7-4426-9921-8d0e626cba52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:35.014377 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fcd03-95c7-4426-9921-8d0e626cba52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:35.015131 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fcd03-95c7-4426-9921-8d0e626cba52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:35.016924 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fcd03-95c7-4426-9921-8d0e626cba52 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:35.027057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 970/1927] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:36:35.031278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25349ea0-ca99-47c3-af67-613ad452347f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:35.041923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25349ea0-ca99-47c3-af67-613ad452347f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:35.047857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1a58b-b085-4c7f-a160-75079537306f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:35.048765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1a58b-b085-4c7f-a160-75079537306f admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:35.049079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1a58b-b085-4c7f-a160-75079537306f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:35.049357 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1a58b-b085-4c7f-a160-75079537306f admin admin] PATH_INFO: `/v3/projects/100b8222e80d43bfa02a27793fa4d859/users/66289f3841e4426b89d8580b7a2ee113/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:35.051558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1a58b-b085-4c7f-a160-75079537306f admin admin] RBAC: Authorizing `identity:create_grant(project_id=100b8222e80d43bfa02a27793fa4d859, user_id=66289f3841e4426b89d8580b7a2ee113, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:35.054837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1a58b-b085-4c7f-a160-75079537306f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:35.076139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 958/1928] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:35 2022] PUT /identity/v3/projects/100b8222e80d43bfa02a27793fa4d859/users/66289f3841e4426b89d8580b7a2ee113/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:35.078617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e36ae4-d7c7-47fb-b2f1-d6dc3dff0041 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:35.078617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e36ae4-d7c7-47fb-b2f1-d6dc3dff0041 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:35.078847 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e36ae4-d7c7-47fb-b2f1-d6dc3dff0041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:35.111415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-51e36ae4-d7c7-47fb-b2f1-d6dc3dff0041 None None] MFA Rules not processed for user `66289f3841e4426b89d8580b7a2ee113`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:35.112666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-51e36ae4-d7c7-47fb-b2f1-d6dc3dff0041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:35.195759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 971/1929] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:35.204097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1a58b-b085-4c7f-a160-75079537306f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:35.206296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-40d1a58b-b085-4c7f-a160-75079537306f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:35.271395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1a58b-b085-4c7f-a160-75079537306f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:35.276115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90b2a3-238b-43fd-a11f-100c5ddb93d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:35.276789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90b2a3-238b-43fd-a11f-100c5ddb93d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:35.277073 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90b2a3-238b-43fd-a11f-100c5ddb93d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:35.277353 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90b2a3-238b-43fd-a11f-100c5ddb93d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:35.288401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90b2a3-238b-43fd-a11f-100c5ddb93d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:35.290030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90b2a3-238b-43fd-a11f-100c5ddb93d7 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:35.305299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 959/1930] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:35.788534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e36ae4-d7c7-47fb-b2f1-d6dc3dff0041 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:35.794354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:35.799021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e36ae4-d7c7-47fb-b2f1-d6dc3dff0041 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:35.804954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eb84b-745d-41a0-a4a9-52fb66b1e73e tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66289f3841e4426b89d8580b7a2ee113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '100b8222e80d43bfa02a27793fa4d859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:35.805848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eb84b-745d-41a0-a4a9-52fb66b1e73e tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:35.806343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eb84b-745d-41a0-a4a9-52fb66b1e73e tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:35.806843 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565eb84b-745d-41a0-a4a9-52fb66b1e73e tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:35.807625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565eb84b-745d-41a0-a4a9-52fb66b1e73e tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:36:35.807923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565eb84b-745d-41a0-a4a9-52fb66b1e73e tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:35.809569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565eb84b-745d-41a0-a4a9-52fb66b1e73e tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:35.822306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 972/1931] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:36:35.826494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90b2a3-238b-43fd-a11f-100c5ddb93d7 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:35.836112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90b2a3-238b-43fd-a11f-100c5ddb93d7 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:35.841306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8abf01-1bc8-40bb-8e1f-3e74e59dd48f tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66289f3841e4426b89d8580b7a2ee113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '100b8222e80d43bfa02a27793fa4d859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:35.841967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8abf01-1bc8-40bb-8e1f-3e74e59dd48f tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:35.842256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8abf01-1bc8-40bb-8e1f-3e74e59dd48f tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:35.842538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8abf01-1bc8-40bb-8e1f-3e74e59dd48f tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:35.843277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8abf01-1bc8-40bb-8e1f-3e74e59dd48f tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:35.845262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8abf01-1bc8-40bb-8e1f-3e74e59dd48f tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:35.873034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 960/1932] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:35 2022] POST /identity/v3/projects => generated 354 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:36:35.878001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eb84b-745d-41a0-a4a9-52fb66b1e73e tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:35.888441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565eb84b-745d-41a0-a4a9-52fb66b1e73e tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:35.954888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9e17e-d058-40e2-b375-a87ac94623a4 tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66289f3841e4426b89d8580b7a2ee113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '100b8222e80d43bfa02a27793fa4d859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:35.955571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9e17e-d058-40e2-b375-a87ac94623a4 tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:35.955862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9e17e-d058-40e2-b375-a87ac94623a4 tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:35.956139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9e17e-d058-40e2-b375-a87ac94623a4 tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:35.956945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9e17e-d058-40e2-b375-a87ac94623a4 tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:36:35.957267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9e17e-d058-40e2-b375-a87ac94623a4 tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:35.958945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9e17e-d058-40e2-b375-a87ac94623a4 tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:35.972500 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 973/1933] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:36:36.016623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8abf01-1bc8-40bb-8e1f-3e74e59dd48f tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:36.033498 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8abf01-1bc8-40bb-8e1f-3e74e59dd48f tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:36.040442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66289f3841e4426b89d8580b7a2ee113', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '100b8222e80d43bfa02a27793fa4d859', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:36.041335 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:36.041665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:36.041948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] PATH_INFO: `/v3/projects/f4544692107e48a6883aeae07db64a9e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:36.043220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f4544692107e48a6883aeae07db64a9e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:36.045205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:36.099240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4544692107e48a6883aeae07db64a9e', 'request_id': 'req-994b2c3b-107e-4053-8178-e0876565e3ba'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:36.106183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4544692107e48a6883aeae07db64a9e', 'request_id': 'req-994b2c3b-107e-4053-8178-e0876565e3ba'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:36.112575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4544692107e48a6883aeae07db64a9e', 'request_id': 'req-994b2c3b-107e-4053-8178-e0876565e3ba'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:36.128256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] The token cache is being invalidate because project f4544692107e48a6883aeae07db64a9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:36.128547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:36.129307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-994b2c3b-107e-4053-8178-e0876565e3ba tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:36.131055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 961/1934] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:36 2022] DELETE /identity/v3/projects/f4544692107e48a6883aeae07db64a9e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:36.349723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9e17e-d058-40e2-b375-a87ac94623a4 tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:36.353158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-87c9e17e-d058-40e2-b375-a87ac94623a4 tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:36.443634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9e17e-d058-40e2-b375-a87ac94623a4 tempest-VolumeQuotaClassesTest-1945532805 tempest-VolumeQuotaClassesTest-1945532805-project-admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:36.524002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:36.524245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:36.525032 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:36.525120 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] PATH_INFO: `/v3/users/255444d4fa644e6192931c97c535b71c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:36.526317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=255444d4fa644e6192931c97c535b71c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:36.528745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:36.591315 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:36.605911 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '255444d4fa644e6192931c97c535b71c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:36.606774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '255444d4fa644e6192931c97c535b71c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:36.621324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '255444d4fa644e6192931c97c535b71c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:36.622160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '255444d4fa644e6192931c97c535b71c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:36.631429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '255444d4fa644e6192931c97c535b71c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:36.647349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 974/1935] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:36 2022] DELETE /identity/v3/users/255444d4fa644e6192931c97c535b71c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:36.651747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5134f9-b927-4e33-8965-7f85b9565382 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:36.652041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5134f9-b927-4e33-8965-7f85b9565382 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:36.652323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5134f9-b927-4e33-8965-7f85b9565382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:36.672758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fa5134f9-b927-4e33-8965-7f85b9565382 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:36.674123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fa5134f9-b927-4e33-8965-7f85b9565382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:36.679831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:36.747247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 962/1936] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:36.758635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:36.760244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:36.835389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67d1e2-d0ac-4cc9-a685-b97cd2cb1699 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:36.839414 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dd802-73f7-44ac-83e5-a8707c6755cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:36.840403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dd802-73f7-44ac-83e5-a8707c6755cc service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:36.840682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dd802-73f7-44ac-83e5-a8707c6755cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:36.840952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dd802-73f7-44ac-83e5-a8707c6755cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:36.850888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748dd802-73f7-44ac-83e5-a8707c6755cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:36.852373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748dd802-73f7-44ac-83e5-a8707c6755cc service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:36.863065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 975/1937] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:37.151076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5134f9-b927-4e33-8965-7f85b9565382 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:37.152751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fa5134f9-b927-4e33-8965-7f85b9565382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:37.181104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5134f9-b927-4e33-8965-7f85b9565382 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:37.188458 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:37.189575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:37.189872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:37.190842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] PATH_INFO: `/v3/projects/99552574d6b44a3d806c66187c0ef46c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:37.192271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] RBAC: Authorizing `identity:delete_project(project_id=99552574d6b44a3d806c66187c0ef46c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:37.194563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:37.249365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99552574d6b44a3d806c66187c0ef46c', 'request_id': 'req-621eac1b-644a-4fb0-a3ec-c3948c96422c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.256216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99552574d6b44a3d806c66187c0ef46c', 'request_id': 'req-621eac1b-644a-4fb0-a3ec-c3948c96422c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.263465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99552574d6b44a3d806c66187c0ef46c', 'request_id': 'req-621eac1b-644a-4fb0-a3ec-c3948c96422c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.279499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] The token cache is being invalidate because project 99552574d6b44a3d806c66187c0ef46c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:37.279831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.280444 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.282142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 963/1938] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:37 2022] DELETE /identity/v3/projects/99552574d6b44a3d806c66187c0ef46c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:37.286266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dd802-73f7-44ac-83e5-a8707c6755cc service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:37.298030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dd802-73f7-44ac-83e5-a8707c6755cc service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:37.298544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:37.299957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:37.356902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fd7dc-3417-4f64-a06c-0861ca58397f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:37.357785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd7dc-3417-4f64-a06c-0861ca58397f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:37.358068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd7dc-3417-4f64-a06c-0861ca58397f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:37.358484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fd7dc-3417-4f64-a06c-0861ca58397f admin admin] PATH_INFO: `/v3/users/66289f3841e4426b89d8580b7a2ee113` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:37.359731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fd7dc-3417-4f64-a06c-0861ca58397f admin admin] RBAC: Authorizing `identity:delete_user(user_id=66289f3841e4426b89d8580b7a2ee113)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:37.361802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fd7dc-3417-4f64-a06c-0861ca58397f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:37.365835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eac1b-644a-4fb0-a3ec-c3948c96422c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:37.370920 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:37.383921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:37.384313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:37.384709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] PATH_INFO: `/v3/users/e00fcdbafeb8405388c6103c4e0f9eb0` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:37.386712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e00fcdbafeb8405388c6103c4e0f9eb0)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:37.389144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:37.446412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-432fd7dc-3417-4f64-a06c-0861ca58397f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66289f3841e4426b89d8580b7a2ee113'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.448095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-432fd7dc-3417-4f64-a06c-0861ca58397f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66289f3841e4426b89d8580b7a2ee113'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.467106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:37.470440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:37.484634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e00fcdbafeb8405388c6103c4e0f9eb0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.485228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-432fd7dc-3417-4f64-a06c-0861ca58397f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66289f3841e4426b89d8580b7a2ee113'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.485642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e00fcdbafeb8405388c6103c4e0f9eb0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.486091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-432fd7dc-3417-4f64-a06c-0861ca58397f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66289f3841e4426b89d8580b7a2ee113'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.496139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-432fd7dc-3417-4f64-a06c-0861ca58397f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66289f3841e4426b89d8580b7a2ee113'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.501531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e00fcdbafeb8405388c6103c4e0f9eb0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.502496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e00fcdbafeb8405388c6103c4e0f9eb0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.509410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 976/1939] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:37 2022] DELETE /identity/v3/users/66289f3841e4426b89d8580b7a2ee113 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:37.511129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e00fcdbafeb8405388c6103c4e0f9eb0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:37.524181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 964/1940] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:37 2022] DELETE /identity/v3/users/e00fcdbafeb8405388c6103c4e0f9eb0 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:37.528837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a612b-5380-4c37-a3b9-9f76e1fe6631 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:37.529214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a612b-5380-4c37-a3b9-9f76e1fe6631 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:37.529560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a612b-5380-4c37-a3b9-9f76e1fe6631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:37.550260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-019a612b-5380-4c37-a3b9-9f76e1fe6631 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:37.551575 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-019a612b-5380-4c37-a3b9-9f76e1fe6631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:37.643621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 977/1941] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:36:37.653392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:37.654941 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:37.736521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c11a3-6e1c-400a-af1c-969bc89c6e43 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:37.740554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d05f6-f3bb-49d3-912a-76ff27522641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:37.741391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d05f6-f3bb-49d3-912a-76ff27522641 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:37.741545 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d05f6-f3bb-49d3-912a-76ff27522641 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:37.741828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d05f6-f3bb-49d3-912a-76ff27522641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:37.755469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d05f6-f3bb-49d3-912a-76ff27522641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:37.757210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d05f6-f3bb-49d3-912a-76ff27522641 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:37.773636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 965/1942] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:38.233936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a612b-5380-4c37-a3b9-9f76e1fe6631 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:38.236187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-019a612b-5380-4c37-a3b9-9f76e1fe6631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:38.251039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a612b-5380-4c37-a3b9-9f76e1fe6631 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:38.252341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d05f6-f3bb-49d3-912a-76ff27522641 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:38.254422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-492d05f6-f3bb-49d3-912a-76ff27522641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:38.259004 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:38.260016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:38.260987 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:38.261372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] PATH_INFO: `/v3/projects/100b8222e80d43bfa02a27793fa4d859` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:38.263197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=100b8222e80d43bfa02a27793fa4d859)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:38.265704 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:38.277561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d05f6-f3bb-49d3-912a-76ff27522641 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:38.283861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:38.284948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:38.285997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:38.286439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] PATH_INFO: `/v3/projects/ed2483111483446dad226d6bee3f6990` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:38.288148 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2483111483446dad226d6bee3f6990)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:38.291270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:38.345432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '100b8222e80d43bfa02a27793fa4d859', 'request_id': 'req-9b7cd118-f041-40ad-a2f6-076ca1705520'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.353705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '100b8222e80d43bfa02a27793fa4d859', 'request_id': 'req-9b7cd118-f041-40ad-a2f6-076ca1705520'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.360833 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '100b8222e80d43bfa02a27793fa4d859', 'request_id': 'req-9b7cd118-f041-40ad-a2f6-076ca1705520'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.388095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] The token cache is being invalidate because project 100b8222e80d43bfa02a27793fa4d859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:38.388391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2483111483446dad226d6bee3f6990', 'request_id': 'req-1cd99faa-6cf4-49c5-8e51-320fa252acec'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.390022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.391712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9b7cd118-f041-40ad-a2f6-076ca1705520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.402088 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2483111483446dad226d6bee3f6990', 'request_id': 'req-1cd99faa-6cf4-49c5-8e51-320fa252acec'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.405535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 978/1943] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:38 2022] DELETE /identity/v3/projects/100b8222e80d43bfa02a27793fa4d859 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:38.414094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c92eca-839e-4afb-a9dc-cbb9524b1cd7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:38.414887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c92eca-839e-4afb-a9dc-cbb9524b1cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:38.415340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c92eca-839e-4afb-a9dc-cbb9524b1cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:38.418393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2483111483446dad226d6bee3f6990', 'request_id': 'req-1cd99faa-6cf4-49c5-8e51-320fa252acec'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.444644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] The token cache is being invalidate because project ed2483111483446dad226d6bee3f6990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:38.445474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.446760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.453440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 966/1944] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:38 2022] DELETE /identity/v3/projects/ed2483111483446dad226d6bee3f6990 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:38.458792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:38.461179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:38.477700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-95c92eca-839e-4afb-a9dc-cbb9524b1cd7 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:38.480156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-95c92eca-839e-4afb-a9dc-cbb9524b1cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:38.542789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:38.612224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd99faa-6cf4-49c5-8e51-320fa252acec admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:38.621458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:38.640963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:38.647205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:38.647205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:38.647205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] PATH_INFO: `/v3/users/e96d41a212304674a0c2be0488f8fc0b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:38.649570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e96d41a212304674a0c2be0488f8fc0b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:38.650467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 979/1945] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:38.654936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c6cdd-5d26-4963-9e59-69b235ff009f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:38.656318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c6cdd-5d26-4963-9e59-69b235ff009f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:38.659198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c6cdd-5d26-4963-9e59-69b235ff009f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:38.660445 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:38.690362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-516c6cdd-5d26-4963-9e59-69b235ff009f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:38.691619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-516c6cdd-5d26-4963-9e59-69b235ff009f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:38.705088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 980/1946] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:38.707607 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c6cdd-5d26-4963-9e59-69b235ff009f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:38.722977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c6cdd-5d26-4963-9e59-69b235ff009f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:38.730479 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c916a0-a9df-4cee-95f6-a5858dd55d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:38.731284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c916a0-a9df-4cee-95f6-a5858dd55d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:38.731626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c916a0-a9df-4cee-95f6-a5858dd55d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:38.731959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c916a0-a9df-4cee-95f6-a5858dd55d13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:38.732797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c916a0-a9df-4cee-95f6-a5858dd55d13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:36:38.733163 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c916a0-a9df-4cee-95f6-a5858dd55d13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:38.735325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c916a0-a9df-4cee-95f6-a5858dd55d13 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:38.737145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e96d41a212304674a0c2be0488f8fc0b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.737911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e96d41a212304674a0c2be0488f8fc0b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.752506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 981/1947] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:36:38.756287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e96d41a212304674a0c2be0488f8fc0b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.757199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c916a0-a9df-4cee-95f6-a5858dd55d13 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:38.760229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-22c916a0-a9df-4cee-95f6-a5858dd55d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:38.771307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e96d41a212304674a0c2be0488f8fc0b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.784775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e96d41a212304674a0c2be0488f8fc0b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:38.802784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c916a0-a9df-4cee-95f6-a5858dd55d13 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:38.811640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da31b15-543f-42de-8fc2-e944d9bfd699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:38.812851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da31b15-543f-42de-8fc2-e944d9bfd699 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:38.813608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da31b15-543f-42de-8fc2-e944d9bfd699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:38.814285 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da31b15-543f-42de-8fc2-e944d9bfd699 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:38.815880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da31b15-543f-42de-8fc2-e944d9bfd699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:38.819477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 967/1948] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:38 2022] DELETE /identity/v3/users/e96d41a212304674a0c2be0488f8fc0b => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:36:38.819954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da31b15-543f-42de-8fc2-e944d9bfd699 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:38.899091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 982/1949] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:38 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:38.903880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:39.055435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051cdc2-fd72-4946-a4a2-6a0b3d8b4ec2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:39.149134 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a70c3-af39-4723-9c7f-4c5678a67272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:39.149923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a70c3-af39-4723-9c7f-4c5678a67272 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:39.150275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a70c3-af39-4723-9c7f-4c5678a67272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:39.150616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a70c3-af39-4723-9c7f-4c5678a67272 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:39.151522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a70c3-af39-4723-9c7f-4c5678a67272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:39.154019 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511a70c3-af39-4723-9c7f-4c5678a67272 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:39.170950 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:39.199261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 968/1950] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:38 2022] POST /identity/v3/users => generated 287 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:39.199980 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da31b15-543f-42de-8fc2-e944d9bfd699 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:39.217540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da31b15-543f-42de-8fc2-e944d9bfd699 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:39.222527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc2fea-0abf-48d5-b6d3-5e303cbfaf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:39.223358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc2fea-0abf-48d5-b6d3-5e303cbfaf83 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:39.223735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc2fea-0abf-48d5-b6d3-5e303cbfaf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:39.224137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc2fea-0abf-48d5-b6d3-5e303cbfaf83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:39.225125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc2fea-0abf-48d5-b6d3-5e303cbfaf83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:39.228093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc2fea-0abf-48d5-b6d3-5e303cbfaf83 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:39.243295 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 983/1951] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:36:39.247063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a70c3-af39-4723-9c7f-4c5678a67272 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:39.261120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511a70c3-af39-4723-9c7f-4c5678a67272 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:39.267564 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb20980-3295-4e24-8a8b-85672c35a68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:39.268278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb20980-3295-4e24-8a8b-85672c35a68b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:39.268544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb20980-3295-4e24-8a8b-85672c35a68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:39.268811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb20980-3295-4e24-8a8b-85672c35a68b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:39.269557 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb20980-3295-4e24-8a8b-85672c35a68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:39.271508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb20980-3295-4e24-8a8b-85672c35a68b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:39.283237 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 969/1952] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:36:39.287611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc2fea-0abf-48d5-b6d3-5e303cbfaf83 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:39.303872 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc2fea-0abf-48d5-b6d3-5e303cbfaf83 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:39.310159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34824f35-827b-46f9-8da8-da8b8afbf8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:39.311814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34824f35-827b-46f9-8da8-da8b8afbf8c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:39.312297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34824f35-827b-46f9-8da8-da8b8afbf8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:39.312784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34824f35-827b-46f9-8da8-da8b8afbf8c8 admin admin] PATH_INFO: `/v3/projects/a39d1fc84c6140a79734b04cd5a70249/users/6bcb4e12327e4922af739cbb6be41624/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:39.337762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34824f35-827b-46f9-8da8-da8b8afbf8c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a39d1fc84c6140a79734b04cd5a70249, user_id=6bcb4e12327e4922af739cbb6be41624, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:39.340887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34824f35-827b-46f9-8da8-da8b8afbf8c8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:39.359800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 984/1953] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:39 2022] PUT /identity/v3/projects/a39d1fc84c6140a79734b04cd5a70249/users/6bcb4e12327e4922af739cbb6be41624/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:39.366877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20cd2a-d936-4252-b749-1f0161632c6e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:39.366877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20cd2a-d936-4252-b749-1f0161632c6e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:39.366877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20cd2a-d936-4252-b749-1f0161632c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:39.391794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2d20cd2a-d936-4252-b749-1f0161632c6e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:39.392947 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2d20cd2a-d936-4252-b749-1f0161632c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:39.478716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 970/1954] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:39.488588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34824f35-827b-46f9-8da8-da8b8afbf8c8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:39.490445 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-34824f35-827b-46f9-8da8-da8b8afbf8c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:39.582560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34824f35-827b-46f9-8da8-da8b8afbf8c8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:39.586308 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20cd2a-d936-4252-b749-1f0161632c6e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:39.587346 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8adb96-aa84-49d0-9509-415456df8724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:39.591433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2d20cd2a-d936-4252-b749-1f0161632c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:39.591520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8adb96-aa84-49d0-9509-415456df8724 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:39.591520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8adb96-aa84-49d0-9509-415456df8724 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:39.591793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8adb96-aa84-49d0-9509-415456df8724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:39.604571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8adb96-aa84-49d0-9509-415456df8724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:39.605345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20cd2a-d936-4252-b749-1f0161632c6e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:39.606242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8adb96-aa84-49d0-9509-415456df8724 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:39.610952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:39.625014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:39.625342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:39.625694 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] PATH_INFO: `/v3/projects/8e24b80d85e54768961b308e7c6bd3ca` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:39.630977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e24b80d85e54768961b308e7c6bd3ca)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:39.632226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 985/1955] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:39.633679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:39.698861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8adb96-aa84-49d0-9509-415456df8724 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:39.700288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-db8adb96-aa84-49d0-9509-415456df8724 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:39.713848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e24b80d85e54768961b308e7c6bd3ca', 'request_id': 'req-ec156269-53bb-4a41-8918-83fd9c358e46'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:39.715575 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8adb96-aa84-49d0-9509-415456df8724 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:39.722578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e24b80d85e54768961b308e7c6bd3ca', 'request_id': 'req-ec156269-53bb-4a41-8918-83fd9c358e46'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:39.731635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e24b80d85e54768961b308e7c6bd3ca', 'request_id': 'req-ec156269-53bb-4a41-8918-83fd9c358e46'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:39.746848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] The token cache is being invalidate because project 8e24b80d85e54768961b308e7c6bd3ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:39.747369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:39.748068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ec156269-53bb-4a41-8918-83fd9c358e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:39.749752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 971/1956] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:39 2022] DELETE /identity/v3/projects/8e24b80d85e54768961b308e7c6bd3ca => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:39.754799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178e49-fab3-4a97-bae6-2bd84493814e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:39.755232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178e49-fab3-4a97-bae6-2bd84493814e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:39.755578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8178e49-fab3-4a97-bae6-2bd84493814e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:39.777397 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a8178e49-fab3-4a97-bae6-2bd84493814e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:39.778715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a8178e49-fab3-4a97-bae6-2bd84493814e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:39.789025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a8e85-610c-4fa6-a3b6-3510d57ce1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:39.789918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a8e85-610c-4fa6-a3b6-3510d57ce1a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:39.790200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a8e85-610c-4fa6-a3b6-3510d57ce1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:39.790474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a8e85-610c-4fa6-a3b6-3510d57ce1a1 admin admin] PATH_INFO: `/v3/users/7d0db1e0495a4cb1ad076985f0cf03e8` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:39.791759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a8e85-610c-4fa6-a3b6-3510d57ce1a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d0db1e0495a4cb1ad076985f0cf03e8)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:39.793700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a8e85-610c-4fa6-a3b6-3510d57ce1a1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:39.829281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:39.856015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-134a8e85-610c-4fa6-a3b6-3510d57ce1a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d0db1e0495a4cb1ad076985f0cf03e8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:39.856791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-134a8e85-610c-4fa6-a3b6-3510d57ce1a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d0db1e0495a4cb1ad076985f0cf03e8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:39.867480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 972/1957] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:39.872866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-134a8e85-610c-4fa6-a3b6-3510d57ce1a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d0db1e0495a4cb1ad076985f0cf03e8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:39.873876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-134a8e85-610c-4fa6-a3b6-3510d57ce1a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d0db1e0495a4cb1ad076985f0cf03e8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:39.875437 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325bb7-4afe-46e1-b05a-7d045e8051d1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:39.875725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325bb7-4afe-46e1-b05a-7d045e8051d1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:39.876157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8325bb7-4afe-46e1-b05a-7d045e8051d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:39.883348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-134a8e85-610c-4fa6-a3b6-3510d57ce1a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d0db1e0495a4cb1ad076985f0cf03e8'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:39.891368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:39.897796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d8325bb7-4afe-46e1-b05a-7d045e8051d1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:39.899101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 986/1958] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:39 2022] DELETE /identity/v3/users/7d0db1e0495a4cb1ad076985f0cf03e8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:39.900064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d8325bb7-4afe-46e1-b05a-7d045e8051d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:39.904071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a3413-f30d-4428-8558-7f57b5862e94 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:39.904363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a3413-f30d-4428-8558-7f57b5862e94 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:39.904752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a3413-f30d-4428-8558-7f57b5862e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:39.923287 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-aa2a3413-f30d-4428-8558-7f57b5862e94 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:39.924637 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-aa2a3413-f30d-4428-8558-7f57b5862e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:39.979045 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 973/1959] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:36:39.983656 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8325bb7-4afe-46e1-b05a-7d045e8051d1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:39.994273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8325bb7-4afe-46e1-b05a-7d045e8051d1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:39.997527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 987/1960] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:40.008499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a3413-f30d-4428-8558-7f57b5862e94 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:40.008960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbfaed2-f7c8-4246-9328-e6f80486e521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:40.009917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfaed2-f7c8-4246-9328-e6f80486e521 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.010061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-aa2a3413-f30d-4428-8558-7f57b5862e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:40.014810 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfaed2-f7c8-4246-9328-e6f80486e521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.015498 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfaed2-f7c8-4246-9328-e6f80486e521 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.016855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbfaed2-f7c8-4246-9328-e6f80486e521 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:36:40.017137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbfaed2-f7c8-4246-9328-e6f80486e521 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:40.019315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbfaed2-f7c8-4246-9328-e6f80486e521 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:40.035412 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 974/1961] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:36:40.042681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbfaed2-f7c8-4246-9328-e6f80486e521 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:40.054550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbfaed2-f7c8-4246-9328-e6f80486e521 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:40.062163 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d60522-00f3-4d53-ad46-ef7b217c3cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:40.063121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d60522-00f3-4d53-ad46-ef7b217c3cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.063469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d60522-00f3-4d53-ad46-ef7b217c3cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.064133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d60522-00f3-4d53-ad46-ef7b217c3cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.064682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d60522-00f3-4d53-ad46-ef7b217c3cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:40.066680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d60522-00f3-4d53-ad46-ef7b217c3cc5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:40.086891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a3413-f30d-4428-8558-7f57b5862e94 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:40.093239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b53c78-2730-4ce3-bd00-913e1565bb1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:40.094238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b53c78-2730-4ce3-bd00-913e1565bb1d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.095943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b53c78-2730-4ce3-bd00-913e1565bb1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.096868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b53c78-2730-4ce3-bd00-913e1565bb1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.112925 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 975/1962] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:40 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:36:40.119144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d60522-00f3-4d53-ad46-ef7b217c3cc5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:40.121634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b53c78-2730-4ce3-bd00-913e1565bb1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:40.124821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b53c78-2730-4ce3-bd00-913e1565bb1d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:40.150926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 988/1963] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:36:40.224353 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d60522-00f3-4d53-ad46-ef7b217c3cc5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:40.232722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbf51e-01b1-4839-a9f1-c6eef5a9d2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:40.233614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbf51e-01b1-4839-a9f1-c6eef5a9d2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.233916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbf51e-01b1-4839-a9f1-c6eef5a9d2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.234343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbf51e-01b1-4839-a9f1-c6eef5a9d2c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.235337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbf51e-01b1-4839-a9f1-c6eef5a9d2c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:40.238049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbf51e-01b1-4839-a9f1-c6eef5a9d2c3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:40.280426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b53c78-2730-4ce3-bd00-913e1565bb1d service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:40.289176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 976/1964] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:40 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:36:40.294292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b53c78-2730-4ce3-bd00-913e1565bb1d service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:40.303055 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ba4c6-a32d-41fb-b2c4-f488b4804dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:40.303950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ba4c6-a32d-41fb-b2c4-f488b4804dff admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.307787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ba4c6-a32d-41fb-b2c4-f488b4804dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.308537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ba4c6-a32d-41fb-b2c4-f488b4804dff admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.310476 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ba4c6-a32d-41fb-b2c4-f488b4804dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:40.314549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ba4c6-a32d-41fb-b2c4-f488b4804dff admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:40.335582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbf51e-01b1-4839-a9f1-c6eef5a9d2c3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:40.348787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbf51e-01b1-4839-a9f1-c6eef5a9d2c3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:40.350177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 989/1965] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:40.357308 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a44b43-5edd-4446-ac8f-d8aad26a2846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:40.358571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44b43-5edd-4446-ac8f-d8aad26a2846 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.358894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44b43-5edd-4446-ac8f-d8aad26a2846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.359175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a44b43-5edd-4446-ac8f-d8aad26a2846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.360744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a44b43-5edd-4446-ac8f-d8aad26a2846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:40.362584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a44b43-5edd-4446-ac8f-d8aad26a2846 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:40.376982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 977/1966] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:40.382581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ba4c6-a32d-41fb-b2c4-f488b4804dff admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:40.410636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ba4c6-a32d-41fb-b2c4-f488b4804dff admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:40.426677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ff22b-69e9-41ef-93f8-20e5ab841a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:40.428710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ff22b-69e9-41ef-93f8-20e5ab841a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.429261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ff22b-69e9-41ef-93f8-20e5ab841a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.429804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ff22b-69e9-41ef-93f8-20e5ab841a2d admin admin] PATH_INFO: `/v3/projects/5ede172a0f1d43ecaa26351640829e7a/users/71b5041864d548ffbd6b30922d418445/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.498494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65449a7a-cd8c-4f47-a817-59e5e0a91889 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.498824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65449a7a-cd8c-4f47-a817-59e5e0a91889 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.499121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65449a7a-cd8c-4f47-a817-59e5e0a91889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.520627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ff22b-69e9-41ef-93f8-20e5ab841a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ede172a0f1d43ecaa26351640829e7a, user_id=71b5041864d548ffbd6b30922d418445, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:40.527529 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ff22b-69e9-41ef-93f8-20e5ab841a2d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:40.564272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 990/1967] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:40 2022] PUT /identity/v3/projects/5ede172a0f1d43ecaa26351640829e7a/users/71b5041864d548ffbd6b30922d418445/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:40.565293 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ff22b-69e9-41ef-93f8-20e5ab841a2d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:40.570047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-603ff22b-69e9-41ef-93f8-20e5ab841a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:40.573861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-65449a7a-cd8c-4f47-a817-59e5e0a91889 None None] MFA Rules not processed for user `6bcb4e12327e4922af739cbb6be41624`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:40.575675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-65449a7a-cd8c-4f47-a817-59e5e0a91889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:40.601376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:40.633825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:40.668025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 978/1968] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:40.673235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a41016-60f8-4fd9-aa81-ae30ca7551e2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.673456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a41016-60f8-4fd9-aa81-ae30ca7551e2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.673639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a41016-60f8-4fd9-aa81-ae30ca7551e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.721719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-48a41016-60f8-4fd9-aa81-ae30ca7551e2 None None] MFA Rules not processed for user `71b5041864d548ffbd6b30922d418445`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:40.723168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ff22b-69e9-41ef-93f8-20e5ab841a2d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:40.723476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-48a41016-60f8-4fd9-aa81-ae30ca7551e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:40.743212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:40.744118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.744398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.744668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] PATH_INFO: `/v3/projects/3682cd9a64cd469684178faa933eb5dd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.746475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3682cd9a64cd469684178faa933eb5dd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:40.750720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:40.838000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3682cd9a64cd469684178faa933eb5dd', 'request_id': 'req-e664561d-72f8-4366-8134-84d9f769e7d0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:40.846785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3682cd9a64cd469684178faa933eb5dd', 'request_id': 'req-e664561d-72f8-4366-8134-84d9f769e7d0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:40.848305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 979/1969] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:40.849151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a41016-60f8-4fd9-aa81-ae30ca7551e2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:40.856977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3682cd9a64cd469684178faa933eb5dd', 'request_id': 'req-e664561d-72f8-4366-8134-84d9f769e7d0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:40.859056 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-48a41016-60f8-4fd9-aa81-ae30ca7551e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:40.874043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] The token cache is being invalidate because project 3682cd9a64cd469684178faa933eb5dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:40.874443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:40.875189 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:40.876982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 991/1970] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:40 2022] DELETE /identity/v3/projects/3682cd9a64cd469684178faa933eb5dd => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:40.877889 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:40.879303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:40.879928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a41016-60f8-4fd9-aa81-ae30ca7551e2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:40.950870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b35f6a-1d9d-4dbf-8f1d-d55d712394ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:40.951266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664561d-72f8-4366-8134-84d9f769e7d0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:40.951548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b35f6a-1d9d-4dbf-8f1d-d55d712394ac admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.951831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b35f6a-1d9d-4dbf-8f1d-d55d712394ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.952107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b35f6a-1d9d-4dbf-8f1d-d55d712394ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.952801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b35f6a-1d9d-4dbf-8f1d-d55d712394ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:40.954883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b35f6a-1d9d-4dbf-8f1d-d55d712394ac admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:40.955674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e55fbf-4933-414c-ac5f-c0ceee3bfadf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:40.956374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e55fbf-4933-414c-ac5f-c0ceee3bfadf service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:40.956654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e55fbf-4933-414c-ac5f-c0ceee3bfadf service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:40.956963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e55fbf-4933-414c-ac5f-c0ceee3bfadf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:40.958845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-82e55fbf-4933-414c-ac5f-c0ceee3bfadf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:40.982412 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 980/1971] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:40 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:40.983549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b35f6a-1d9d-4dbf-8f1d-d55d712394ac admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:40.985181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-60b35f6a-1d9d-4dbf-8f1d-d55d712394ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:41.019139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:41.023022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e55fbf-4933-414c-ac5f-c0ceee3bfadf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:41.023900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e55fbf-4933-414c-ac5f-c0ceee3bfadf service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:41.037434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 992/1972] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:41.038296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e55fbf-4933-414c-ac5f-c0ceee3bfadf service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:41.041217 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-82e55fbf-4933-414c-ac5f-c0ceee3bfadf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:41.056410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:41.062904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e55fbf-4933-414c-ac5f-c0ceee3bfadf service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:41.067706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa529ef-6122-46ea-8190-311cf6c267bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:41.068440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa529ef-6122-46ea-8190-311cf6c267bf admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:41.068728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa529ef-6122-46ea-8190-311cf6c267bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:41.069023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa529ef-6122-46ea-8190-311cf6c267bf admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:41.069791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa529ef-6122-46ea-8190-311cf6c267bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:41.073203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa529ef-6122-46ea-8190-311cf6c267bf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:41.075407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b35f6a-1d9d-4dbf-8f1d-d55d712394ac admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:41.081313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:41.083677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:41.083965 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:41.084694 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] PATH_INFO: `/v3/users/f7196aa1e6ee4809a8a2a15d62cc7053` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:41.086164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7196aa1e6ee4809a8a2a15d62cc7053)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:41.088480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:41.101876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 993/1973] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:41 2022] POST /identity/v3/users => generated 293 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:41.102639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa529ef-6122-46ea-8190-311cf6c267bf admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:41.112944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa529ef-6122-46ea-8190-311cf6c267bf admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:41.118094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010978da-c8db-4dea-a536-18fe830daac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:41.119494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010978da-c8db-4dea-a536-18fe830daac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:41.119792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010978da-c8db-4dea-a536-18fe830daac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:41.120074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010978da-c8db-4dea-a536-18fe830daac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:41.120818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010978da-c8db-4dea-a536-18fe830daac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:41.122440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010978da-c8db-4dea-a536-18fe830daac7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:41.134796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 994/1974] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:41.139316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010978da-c8db-4dea-a536-18fe830daac7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:41.150192 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010978da-c8db-4dea-a536-18fe830daac7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:41.155935 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3620f98-cb95-42fb-93f6-36c9b492464b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:41.156554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3620f98-cb95-42fb-93f6-36c9b492464b admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:41.157032 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3620f98-cb95-42fb-93f6-36c9b492464b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:41.157276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3620f98-cb95-42fb-93f6-36c9b492464b admin admin] PATH_INFO: `/v3/projects/2721b1f85f894eea8171447b77d329b3/users/9fc1fb105ba54d839421ab3f4dccd64c/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:41.181528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3620f98-cb95-42fb-93f6-36c9b492464b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2721b1f85f894eea8171447b77d329b3, user_id=9fc1fb105ba54d839421ab3f4dccd64c, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:41.181728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7196aa1e6ee4809a8a2a15d62cc7053'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:41.182318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7196aa1e6ee4809a8a2a15d62cc7053'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:41.185845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3620f98-cb95-42fb-93f6-36c9b492464b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:41.203053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7196aa1e6ee4809a8a2a15d62cc7053'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:41.204033 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7196aa1e6ee4809a8a2a15d62cc7053'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:41.208044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 995/1975] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:41 2022] PUT /identity/v3/projects/2721b1f85f894eea8171447b77d329b3/users/9fc1fb105ba54d839421ab3f4dccd64c/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:36:41.209419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3620f98-cb95-42fb-93f6-36c9b492464b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:41.212149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a3620f98-cb95-42fb-93f6-36c9b492464b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:41.218680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7196aa1e6ee4809a8a2a15d62cc7053'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:41.251052 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 981/1976] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:40 2022] DELETE /identity/v3/users/f7196aa1e6ee4809a8a2a15d62cc7053 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:41.318255 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3620f98-cb95-42fb-93f6-36c9b492464b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:41.412129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757b41d-4ea8-4aea-ad6b-91877fb415ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:41.412943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757b41d-4ea8-4aea-ad6b-91877fb415ff admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:41.413320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757b41d-4ea8-4aea-ad6b-91877fb415ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:41.413943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757b41d-4ea8-4aea-ad6b-91877fb415ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:41.415530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757b41d-4ea8-4aea-ad6b-91877fb415ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:41.417817 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757b41d-4ea8-4aea-ad6b-91877fb415ff admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:41.436059 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 996/1977] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:36:41.438256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:41.458845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e923e6c-8039-4c68-9036-e4c5152d7bda admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:41.469889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c75bf-35ac-4496-9b95-980b901f23c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:41.471860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c75bf-35ac-4496-9b95-980b901f23c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:41.474624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c75bf-35ac-4496-9b95-980b901f23c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:41.475037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c75bf-35ac-4496-9b95-980b901f23c8 admin admin] PATH_INFO: `/v3/projects/2721b1f85f894eea8171447b77d329b3/users/9fc1fb105ba54d839421ab3f4dccd64c/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:41.482766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8c75bf-35ac-4496-9b95-980b901f23c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2721b1f85f894eea8171447b77d329b3, user_id=9fc1fb105ba54d839421ab3f4dccd64c, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:41.485731 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8c75bf-35ac-4496-9b95-980b901f23c8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:41.499400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 982/1978] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:41 2022] PUT /identity/v3/projects/2721b1f85f894eea8171447b77d329b3/users/9fc1fb105ba54d839421ab3f4dccd64c/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:41.608616 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757b41d-4ea8-4aea-ad6b-91877fb415ff admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:41.622957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a757b41d-4ea8-4aea-ad6b-91877fb415ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:41.646241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757b41d-4ea8-4aea-ad6b-91877fb415ff admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:41.715150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:41.749156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:41.751032 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:41.751740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:41.752218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] PATH_INFO: `/v3/projects/d4c43fbf2a64457da1df8ad944af013b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:41.754554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4c43fbf2a64457da1df8ad944af013b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:41.757861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:41.829719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4c43fbf2a64457da1df8ad944af013b', 'request_id': 'req-40c2c86a-69c2-4bbf-8352-0d73ac04e654'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:41.840657 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4c43fbf2a64457da1df8ad944af013b', 'request_id': 'req-40c2c86a-69c2-4bbf-8352-0d73ac04e654'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:41.848859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4c43fbf2a64457da1df8ad944af013b', 'request_id': 'req-40c2c86a-69c2-4bbf-8352-0d73ac04e654'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:41.877904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] The token cache is being invalidate because project d4c43fbf2a64457da1df8ad944af013b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:41.878268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:41.884658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-40c2c86a-69c2-4bbf-8352-0d73ac04e654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:41.886627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 997/1979] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:41 2022] DELETE /identity/v3/projects/d4c43fbf2a64457da1df8ad944af013b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:41.892525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d56428-1db4-4087-a90f-72e3f8094af1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:41.892786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d56428-1db4-4087-a90f-72e3f8094af1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:41.893068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d56428-1db4-4087-a90f-72e3f8094af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:41.919493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b3d56428-1db4-4087-a90f-72e3f8094af1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:41.920913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b3d56428-1db4-4087-a90f-72e3f8094af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:42.011798 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 983/1980] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:36:42.013147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b595-3e0a-4ec3-806b-d421bb2d7693 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.013502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b595-3e0a-4ec3-806b-d421bb2d7693 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.013847 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451b595-3e0a-4ec3-806b-d421bb2d7693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.040955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f451b595-3e0a-4ec3-806b-d421bb2d7693 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:42.042497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f451b595-3e0a-4ec3-806b-d421bb2d7693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:42.052342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 998/1981] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:36:42.056356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d56428-1db4-4087-a90f-72e3f8094af1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:42.068472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d56428-1db4-4087-a90f-72e3f8094af1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:42.074623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8351c-750a-438f-a923-0f748d9545f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:42.076124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8351c-750a-438f-a923-0f748d9545f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.077219 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8351c-750a-438f-a923-0f748d9545f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.077710 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8351c-750a-438f-a923-0f748d9545f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.078918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8351c-750a-438f-a923-0f748d9545f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:36:42.080126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8351c-750a-438f-a923-0f748d9545f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:42.082259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8351c-750a-438f-a923-0f748d9545f2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:42.101826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 984/1982] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:36:42.107713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451b595-3e0a-4ec3-806b-d421bb2d7693 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:42.119230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451b595-3e0a-4ec3-806b-d421bb2d7693 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:42.124647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0d87c-8b2d-4b0e-bcd9-da9091c1b65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:42.125441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0d87c-8b2d-4b0e-bcd9-da9091c1b65c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.125838 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0d87c-8b2d-4b0e-bcd9-da9091c1b65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.126230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0d87c-8b2d-4b0e-bcd9-da9091c1b65c admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.127736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0d87c-8b2d-4b0e-bcd9-da9091c1b65c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:42.130059 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0d87c-8b2d-4b0e-bcd9-da9091c1b65c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:42.181962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 999/1983] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:42 2022] POST /identity/v3/projects => generated 389 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:42.182176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8351c-750a-438f-a923-0f748d9545f2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:42.189380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8351c-750a-438f-a923-0f748d9545f2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:42.257298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:42.274577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fae49-95e4-4c4c-b1b3-e1234886639c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:42.275771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fae49-95e4-4c4c-b1b3-e1234886639c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.276305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fae49-95e4-4c4c-b1b3-e1234886639c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.276762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fae49-95e4-4c4c-b1b3-e1234886639c admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.277630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0fae49-95e4-4c4c-b1b3-e1234886639c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:42.280013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0fae49-95e4-4c4c-b1b3-e1234886639c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:42.310168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 985/1984] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:42 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:36:42.311388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0d87c-8b2d-4b0e-bcd9-da9091c1b65c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:42.323554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0d87c-8b2d-4b0e-bcd9-da9091c1b65c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:42.334196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd05931-757b-4658-8279-db8f341b24e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:42.335676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05931-757b-4658-8279-db8f341b24e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.336276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05931-757b-4658-8279-db8f341b24e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.336752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd05931-757b-4658-8279-db8f341b24e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.338022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd05931-757b-4658-8279-db8f341b24e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:42.340188 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd05931-757b-4658-8279-db8f341b24e5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:42.353865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1000/1985] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:42.358224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fae49-95e4-4c4c-b1b3-e1234886639c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:42.369600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fae49-95e4-4c4c-b1b3-e1234886639c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:42.374577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670fe1f-ba2e-4467-b229-fd6844b7862e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:42.375497 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670fe1f-ba2e-4467-b229-fd6844b7862e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.376087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670fe1f-ba2e-4467-b229-fd6844b7862e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.378751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670fe1f-ba2e-4467-b229-fd6844b7862e admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.378751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670fe1f-ba2e-4467-b229-fd6844b7862e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:42.379388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670fe1f-ba2e-4467-b229-fd6844b7862e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:42.400186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 986/1986] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:42.404091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd05931-757b-4658-8279-db8f341b24e5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:42.415684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd05931-757b-4658-8279-db8f341b24e5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:42.423903 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bbd8a-fbda-47bd-8966-ef239f22b7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:42.425273 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bbd8a-fbda-47bd-8966-ef239f22b7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.425758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bbd8a-fbda-47bd-8966-ef239f22b7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.426191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bbd8a-fbda-47bd-8966-ef239f22b7c6 admin admin] PATH_INFO: `/v3/projects/c76b27d4765b47e8851a2a59e0016347/users/a556baba9bbd4f569abe243e04fb7069/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.433439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadc3f8-4ef4-4c74-8873-41b4f17e4863 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.434246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadc3f8-4ef4-4c74-8873-41b4f17e4863 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.434246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadc3f8-4ef4-4c74-8873-41b4f17e4863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.448401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bbd8a-fbda-47bd-8966-ef239f22b7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76b27d4765b47e8851a2a59e0016347, user_id=a556baba9bbd4f569abe243e04fb7069, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:42.452941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bbd8a-fbda-47bd-8966-ef239f22b7c6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:42.474765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1001/1987] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:42 2022] PUT /identity/v3/projects/c76b27d4765b47e8851a2a59e0016347/users/a556baba9bbd4f569abe243e04fb7069/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:42.483634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7fadc3f8-4ef4-4c74-8873-41b4f17e4863 None None] MFA Rules not processed for user `9fc1fb105ba54d839421ab3f4dccd64c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:42.483634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7fadc3f8-4ef4-4c74-8873-41b4f17e4863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:42.486987 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf1922-8c66-4baf-920e-f8d455c1472e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.487284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf1922-8c66-4baf-920e-f8d455c1472e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.487562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf1922-8c66-4baf-920e-f8d455c1472e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.568448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a5bf1922-8c66-4baf-920e-f8d455c1472e None None] MFA Rules not processed for user `a556baba9bbd4f569abe243e04fb7069`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:42.569795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a5bf1922-8c66-4baf-920e-f8d455c1472e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:42.577359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 987/1988] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:36:42.603030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadc3f8-4ef4-4c74-8873-41b4f17e4863 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:42.604572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7fadc3f8-4ef4-4c74-8873-41b4f17e4863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:42.661076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1002/1989] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:36:42.665740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf1922-8c66-4baf-920e-f8d455c1472e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:42.672787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:42.695047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadc3f8-4ef4-4c74-8873-41b4f17e4863 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:42.698955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405027ab-7d26-435f-9e9e-0900e27e2db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:42.699627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405027ab-7d26-435f-9e9e-0900e27e2db0 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.699908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405027ab-7d26-435f-9e9e-0900e27e2db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.700347 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405027ab-7d26-435f-9e9e-0900e27e2db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.702263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-405027ab-7d26-435f-9e9e-0900e27e2db0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:42.747933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf1922-8c66-4baf-920e-f8d455c1472e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:42.752956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7d43e-a86d-416f-8e8b-3fca162ad7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:42.753638 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d43e-a86d-416f-8e8b-3fca162ad7f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.753990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d43e-a86d-416f-8e8b-3fca162ad7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.754333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7d43e-a86d-416f-8e8b-3fca162ad7f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.755134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7d43e-a86d-416f-8e8b-3fca162ad7f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:42.757165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7d43e-a86d-416f-8e8b-3fca162ad7f6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:42.782934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405027ab-7d26-435f-9e9e-0900e27e2db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:42.786263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405027ab-7d26-435f-9e9e-0900e27e2db0 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:42.789402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1003/1990] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:42 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:42.793761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7d43e-a86d-416f-8e8b-3fca162ad7f6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:42.799725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 988/1991] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:36:42.862245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7d43e-a86d-416f-8e8b-3fca162ad7f6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:42.869342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb326494-8fa3-4183-bfb1-37c93fee046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:42.870166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb326494-8fa3-4183-bfb1-37c93fee046b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.870680 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb326494-8fa3-4183-bfb1-37c93fee046b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.870960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb326494-8fa3-4183-bfb1-37c93fee046b admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.871777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb326494-8fa3-4183-bfb1-37c93fee046b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:42.874228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb326494-8fa3-4183-bfb1-37c93fee046b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:42.917213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405027ab-7d26-435f-9e9e-0900e27e2db0 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:42.924357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1004/1992] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:42 2022] POST /identity/v3/users => generated 300 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:42.936721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405027ab-7d26-435f-9e9e-0900e27e2db0 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:42.947245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8d7ff-f453-4265-a617-6c61c74c4e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:42.949509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8d7ff-f453-4265-a617-6c61c74c4e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:42.950495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8d7ff-f453-4265-a617-6c61c74c4e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:42.951263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade8d7ff-f453-4265-a617-6c61c74c4e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:42.953770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8d7ff-f453-4265-a617-6c61c74c4e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:42.956957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade8d7ff-f453-4265-a617-6c61c74c4e92 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:42.964318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:42.990850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 989/1993] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:36:42.995128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb326494-8fa3-4183-bfb1-37c93fee046b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:43.008871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb326494-8fa3-4183-bfb1-37c93fee046b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:43.013843 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd801e-e6c3-4638-842d-66529053ed74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:43.014816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd801e-e6c3-4638-842d-66529053ed74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:43.015099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd801e-e6c3-4638-842d-66529053ed74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:43.015376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd801e-e6c3-4638-842d-66529053ed74 admin admin] PATH_INFO: `/v3/projects/0c91fd299679449d85006bb4e726a562/users/c19c5e0709ca4c42bd41257f8ca0c83d/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:43.035469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd801e-e6c3-4638-842d-66529053ed74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c91fd299679449d85006bb4e726a562, user_id=c19c5e0709ca4c42bd41257f8ca0c83d, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:43.038450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfd801e-e6c3-4638-842d-66529053ed74 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:43.059916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1005/1994] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:42 2022] PUT /identity/v3/projects/0c91fd299679449d85006bb4e726a562/users/c19c5e0709ca4c42bd41257f8ca0c83d/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:43.063918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8d7ff-f453-4265-a617-6c61c74c4e92 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:43.076242 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade8d7ff-f453-4265-a617-6c61c74c4e92 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:43.163079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f311016-e7c1-4a17-b522-13706ee6aaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:43.163902 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f311016-e7c1-4a17-b522-13706ee6aaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:43.164301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f311016-e7c1-4a17-b522-13706ee6aaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:43.164889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f311016-e7c1-4a17-b522-13706ee6aaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:43.165770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f311016-e7c1-4a17-b522-13706ee6aaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:43.167663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f311016-e7c1-4a17-b522-13706ee6aaa1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:43.182409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 990/1995] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:43.186466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd801e-e6c3-4638-842d-66529053ed74 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:43.208035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd801e-e6c3-4638-842d-66529053ed74 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:43.227761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044e18e-e175-414c-a6f1-d9f9668a2f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:43.228876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044e18e-e175-414c-a6f1-d9f9668a2f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:43.229296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044e18e-e175-414c-a6f1-d9f9668a2f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:43.229718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044e18e-e175-414c-a6f1-d9f9668a2f04 admin admin] PATH_INFO: `/v3/projects/0c91fd299679449d85006bb4e726a562/users/c19c5e0709ca4c42bd41257f8ca0c83d/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:43.231926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044e18e-e175-414c-a6f1-d9f9668a2f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c91fd299679449d85006bb4e726a562, user_id=c19c5e0709ca4c42bd41257f8ca0c83d, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:43.234808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044e18e-e175-414c-a6f1-d9f9668a2f04 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:43.260127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1006/1996] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:43 2022] PUT /identity/v3/projects/0c91fd299679449d85006bb4e726a562/users/c19c5e0709ca4c42bd41257f8ca0c83d/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:43.261168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dcf91-7450-4d8d-9189-68c6a5176f34 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:43.261464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dcf91-7450-4d8d-9189-68c6a5176f34 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:43.261747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277dcf91-7450-4d8d-9189-68c6a5176f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:43.302579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-277dcf91-7450-4d8d-9189-68c6a5176f34 None None] MFA Rules not processed for user `c19c5e0709ca4c42bd41257f8ca0c83d`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:43.304415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-277dcf91-7450-4d8d-9189-68c6a5176f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:43.384712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 991/1997] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:43.395495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044e18e-e175-414c-a6f1-d9f9668a2f04 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:43.460614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:43.477345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044e18e-e175-414c-a6f1-d9f9668a2f04 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:43.481366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baad2a2-0ad8-4a43-98ad-ea0a7eb7c888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:43.482170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baad2a2-0ad8-4a43-98ad-ea0a7eb7c888 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:43.482511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baad2a2-0ad8-4a43-98ad-ea0a7eb7c888 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:43.482919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baad2a2-0ad8-4a43-98ad-ea0a7eb7c888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:43.494291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baad2a2-0ad8-4a43-98ad-ea0a7eb7c888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:43.495876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baad2a2-0ad8-4a43-98ad-ea0a7eb7c888 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:43.506170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1007/1998] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:43.642832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dcf91-7450-4d8d-9189-68c6a5176f34 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:43.725299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277dcf91-7450-4d8d-9189-68c6a5176f34 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:43.730525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:43.732853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:43.733310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:43.733709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] PATH_INFO: `/v3/users/a556baba9bbd4f569abe243e04fb7069` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:43.735304 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a556baba9bbd4f569abe243e04fb7069)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:43.737496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:43.770356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:43.812151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a556baba9bbd4f569abe243e04fb7069'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:43.813139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a556baba9bbd4f569abe243e04fb7069'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:43.831101 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a556baba9bbd4f569abe243e04fb7069'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:43.832585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a556baba9bbd4f569abe243e04fb7069'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:43.852761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a556baba9bbd4f569abe243e04fb7069'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:43.872064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 992/1999] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:43 2022] DELETE /identity/v3/users/a556baba9bbd4f569abe243e04fb7069 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:43.876690 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a487d-ba41-4352-b2a6-f2ed16fb5388 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:43.878400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a487d-ba41-4352-b2a6-f2ed16fb5388 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:43.878400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a487d-ba41-4352-b2a6-f2ed16fb5388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:43.906400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5a1a487d-ba41-4352-b2a6-f2ed16fb5388 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:43.907813 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5a1a487d-ba41-4352-b2a6-f2ed16fb5388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:43.999671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1008/2000] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:36:44.004386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:44.007505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:44.110050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a71357-df35-471c-8527-ba798de3c2e0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:44.115453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a4352-c1be-42b6-a640-a75c66e5c9da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:44.116070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a4352-c1be-42b6-a640-a75c66e5c9da service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:44.116525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a4352-c1be-42b6-a640-a75c66e5c9da service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:44.116816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a4352-c1be-42b6-a640-a75c66e5c9da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:44.147223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a4352-c1be-42b6-a640-a75c66e5c9da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:44.148781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a4352-c1be-42b6-a640-a75c66e5c9da service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:44.160129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 993/2001] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:44.549718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a487d-ba41-4352-b2a6-f2ed16fb5388 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:44.551207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5a1a487d-ba41-4352-b2a6-f2ed16fb5388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:44.569681 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a487d-ba41-4352-b2a6-f2ed16fb5388 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:44.583981 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:44.584878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:44.585156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:44.585830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] PATH_INFO: `/v3/projects/c76b27d4765b47e8851a2a59e0016347` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:44.586676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c76b27d4765b47e8851a2a59e0016347)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:44.588632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:44.641351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c76b27d4765b47e8851a2a59e0016347', 'request_id': 'req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:44.648289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c76b27d4765b47e8851a2a59e0016347', 'request_id': 'req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:44.657204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c76b27d4765b47e8851a2a59e0016347', 'request_id': 'req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:44.680132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] The token cache is being invalidate because project c76b27d4765b47e8851a2a59e0016347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:44.680426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:44.681089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:44.682927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1009/2002] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:44 2022] DELETE /identity/v3/projects/c76b27d4765b47e8851a2a59e0016347 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:44.686728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a4352-c1be-42b6-a640-a75c66e5c9da service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:44.700216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a4352-c1be-42b6-a640-a75c66e5c9da service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:44.792561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:44.793647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:44.794025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:44.794450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] PATH_INFO: `/v3/users/c19c5e0709ca4c42bd41257f8ca0c83d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:44.795915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c19c5e0709ca4c42bd41257f8ca0c83d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:44.800032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:44.870441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:44.878636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c19c5e0709ca4c42bd41257f8ca0c83d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:44.879556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c19c5e0709ca4c42bd41257f8ca0c83d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:44.899317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c19c5e0709ca4c42bd41257f8ca0c83d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:44.900587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c19c5e0709ca4c42bd41257f8ca0c83d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:44.915936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c19c5e0709ca4c42bd41257f8ca0c83d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:44.933456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 994/2003] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:36:44 2022] DELETE /identity/v3/users/c19c5e0709ca4c42bd41257f8ca0c83d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:45.214431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:45.216824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:45.233962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12575fc-e7b7-4d3b-9572-ca1a1d4fed77 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:45.298346 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:45.299267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:45.299549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:45.299822 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] PATH_INFO: `/v3/projects/0c91fd299679449d85006bb4e726a562` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:45.304109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c91fd299679449d85006bb4e726a562)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:45.306043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:45.314941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:45.362833 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c91fd299679449d85006bb4e726a562', 'request_id': 'req-1fc79806-a58b-4734-ab3d-1996b86ea332'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:45.369426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c91fd299679449d85006bb4e726a562', 'request_id': 'req-1fc79806-a58b-4734-ab3d-1996b86ea332'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:45.369808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:45.371289 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:45.374896 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c91fd299679449d85006bb4e726a562', 'request_id': 'req-1fc79806-a58b-4734-ab3d-1996b86ea332'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:45.389520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] The token cache is being invalidate because project 0c91fd299679449d85006bb4e726a562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:36:45.389802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:45.390472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1fc79806-a58b-4734-ab3d-1996b86ea332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:36:45.392302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1010/2004] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:36:45 2022] DELETE /identity/v3/projects/0c91fd299679449d85006bb4e726a562 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:45.397184 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f541cac-d7b3-4d5d-b0c4-62f48ecba621 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:45.397460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f541cac-d7b3-4d5d-b0c4-62f48ecba621 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:45.397762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f541cac-d7b3-4d5d-b0c4-62f48ecba621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:45.416012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6f541cac-d7b3-4d5d-b0c4-62f48ecba621 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:45.417414 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6f541cac-d7b3-4d5d-b0c4-62f48ecba621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:45.430588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde819d0-7163-48ae-9dcb-dbd712ebe849 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:45.478165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1011/2005] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:36:45.483576 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cb60a-512d-44b7-b866-d2f3448241ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:45.484265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cb60a-512d-44b7-b866-d2f3448241ee service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:45.484548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cb60a-512d-44b7-b866-d2f3448241ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:45.485129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cb60a-512d-44b7-b866-d2f3448241ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:45.485583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97334d49-88a5-4470-b8ee-b4bc86f8c02a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:45.485874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97334d49-88a5-4470-b8ee-b4bc86f8c02a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:45.486149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97334d49-88a5-4470-b8ee-b4bc86f8c02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:45.487391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bc9cb60a-512d-44b7-b866-d2f3448241ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:45.507693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-97334d49-88a5-4470-b8ee-b4bc86f8c02a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:45.508913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-97334d49-88a5-4470-b8ee-b4bc86f8c02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:45.518776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1012/2006] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:36:45.526239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97334d49-88a5-4470-b8ee-b4bc86f8c02a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:45.536378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97334d49-88a5-4470-b8ee-b4bc86f8c02a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:45.541073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cff551-b0f5-4287-9aa0-2f2deaf3ec97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:45.541730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cff551-b0f5-4287-9aa0-2f2deaf3ec97 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:45.542014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cff551-b0f5-4287-9aa0-2f2deaf3ec97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:45.542288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cff551-b0f5-4287-9aa0-2f2deaf3ec97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:45.543057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cff551-b0f5-4287-9aa0-2f2deaf3ec97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:36:45.543367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cff551-b0f5-4287-9aa0-2f2deaf3ec97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:45.545010 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cff551-b0f5-4287-9aa0-2f2deaf3ec97 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:45.559929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1013/2007] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:36:45.560541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cb60a-512d-44b7-b866-d2f3448241ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:45.560867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cb60a-512d-44b7-b866-d2f3448241ee service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:45.561139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cff551-b0f5-4287-9aa0-2f2deaf3ec97 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:45.570899 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cff551-b0f5-4287-9aa0-2f2deaf3ec97 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:45.571313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 995/2008] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:45.575810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df67dc9-6106-48a9-ba1c-2d9680b599e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:45.576457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df67dc9-6106-48a9-ba1c-2d9680b599e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:45.576743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df67dc9-6106-48a9-ba1c-2d9680b599e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:45.577078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df67dc9-6106-48a9-ba1c-2d9680b599e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:45.577714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df67dc9-6106-48a9-ba1c-2d9680b599e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:45.579768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df67dc9-6106-48a9-ba1c-2d9680b599e7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:45.608783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1014/2009] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:45 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:36:45.613681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cb60a-512d-44b7-b866-d2f3448241ee service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:45.631444 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cb60a-512d-44b7-b866-d2f3448241ee service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:45.654820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:45.709386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f6452-17dc-4db0-a9c6-05e567e08c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:45.710162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f6452-17dc-4db0-a9c6-05e567e08c2b admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:45.710509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f6452-17dc-4db0-a9c6-05e567e08c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:45.710873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f6452-17dc-4db0-a9c6-05e567e08c2b admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:45.711684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f6452-17dc-4db0-a9c6-05e567e08c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:45.713659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f6452-17dc-4db0-a9c6-05e567e08c2b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:45.739052 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 996/2010] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:45 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:45.743588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df67dc9-6106-48a9-ba1c-2d9680b599e7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:45.754955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df67dc9-6106-48a9-ba1c-2d9680b599e7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:45.762045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355b3b5-4739-475c-aba0-876f2856dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:45.762851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355b3b5-4739-475c-aba0-876f2856dc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:45.763197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355b3b5-4739-475c-aba0-876f2856dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:45.763577 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355b3b5-4739-475c-aba0-876f2856dc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:45.764459 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355b3b5-4739-475c-aba0-876f2856dc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:45.766269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355b3b5-4739-475c-aba0-876f2856dc3d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:45.777582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1015/2011] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:36:45.781611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f6452-17dc-4db0-a9c6-05e567e08c2b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:45.792292 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f6452-17dc-4db0-a9c6-05e567e08c2b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:45.796957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9319b6e0-4b84-42b3-a285-241ad30b9520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:45.797819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319b6e0-4b84-42b3-a285-241ad30b9520 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:45.798104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319b6e0-4b84-42b3-a285-241ad30b9520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:45.798385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9319b6e0-4b84-42b3-a285-241ad30b9520 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:45.799190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9319b6e0-4b84-42b3-a285-241ad30b9520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:45.800911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9319b6e0-4b84-42b3-a285-241ad30b9520 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:45.811305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 997/2012] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:36:45.815728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355b3b5-4739-475c-aba0-876f2856dc3d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:45.825829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355b3b5-4739-475c-aba0-876f2856dc3d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:45.830577 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93c2d0-47f4-46b5-bab7-79a0ad31dbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:45.831435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93c2d0-47f4-46b5-bab7-79a0ad31dbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:45.831716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93c2d0-47f4-46b5-bab7-79a0ad31dbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:45.831989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93c2d0-47f4-46b5-bab7-79a0ad31dbf2 admin admin] PATH_INFO: `/v3/projects/9bf344fd1761412a9c4601592b43cd0c/users/4802ea8a54f64f53ae4b18bded752836/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:45.847438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93c2d0-47f4-46b5-bab7-79a0ad31dbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf344fd1761412a9c4601592b43cd0c, user_id=4802ea8a54f64f53ae4b18bded752836, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:45.850091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93c2d0-47f4-46b5-bab7-79a0ad31dbf2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:45.865706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1016/2013] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:45 2022] PUT /identity/v3/projects/9bf344fd1761412a9c4601592b43cd0c/users/4802ea8a54f64f53ae4b18bded752836/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:36:45.876098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35de79-6f1e-4ee4-9a3c-a289be9ff42d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:45.876392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35de79-6f1e-4ee4-9a3c-a289be9ff42d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:45.876682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35de79-6f1e-4ee4-9a3c-a289be9ff42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:45.904852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7e35de79-6f1e-4ee4-9a3c-a289be9ff42d None None] MFA Rules not processed for user `4802ea8a54f64f53ae4b18bded752836`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:45.906098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7e35de79-6f1e-4ee4-9a3c-a289be9ff42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:45.986682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 998/2014] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:45.991219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93c2d0-47f4-46b5-bab7-79a0ad31dbf2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:46.012945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:46.057249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93c2d0-47f4-46b5-bab7-79a0ad31dbf2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:46.061881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fa358-938d-46bc-bfbf-867fc73804bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:46.062547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fa358-938d-46bc-bfbf-867fc73804bc admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.062909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fa358-938d-46bc-bfbf-867fc73804bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.063309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fa358-938d-46bc-bfbf-867fc73804bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.064358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fa358-938d-46bc-bfbf-867fc73804bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:46.066270 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fa358-938d-46bc-bfbf-867fc73804bc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:46.094428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1017/2015] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:45 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:46.098847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35de79-6f1e-4ee4-9a3c-a289be9ff42d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:46.109904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35de79-6f1e-4ee4-9a3c-a289be9ff42d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:46.168142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ecc3f-e9b1-4c52-bf43-556f2266f8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:46.168831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ecc3f-e9b1-4c52-bf43-556f2266f8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.169115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ecc3f-e9b1-4c52-bf43-556f2266f8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.169390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ecc3f-e9b1-4c52-bf43-556f2266f8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.170095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ecc3f-e9b1-4c52-bf43-556f2266f8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:46.172044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ecc3f-e9b1-4c52-bf43-556f2266f8a0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:46.194563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 999/2016] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:46 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:36:46.198650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fa358-938d-46bc-bfbf-867fc73804bc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:46.209957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fa358-938d-46bc-bfbf-867fc73804bc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:46.215772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bdfc2-7f38-4f78-b545-6e3b513bc756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:46.216473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bdfc2-7f38-4f78-b545-6e3b513bc756 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.216825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bdfc2-7f38-4f78-b545-6e3b513bc756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.217023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bdfc2-7f38-4f78-b545-6e3b513bc756 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.217757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bdfc2-7f38-4f78-b545-6e3b513bc756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:46.219442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bdfc2-7f38-4f78-b545-6e3b513bc756 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:46.230165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1018/2017] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:46.234155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ecc3f-e9b1-4c52-bf43-556f2266f8a0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:46.244572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ecc3f-e9b1-4c52-bf43-556f2266f8a0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:46.249995 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefb9fc-8e45-4e3d-8466-064bda6e780b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:46.251819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefb9fc-8e45-4e3d-8466-064bda6e780b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.252322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefb9fc-8e45-4e3d-8466-064bda6e780b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.252592 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefb9fc-8e45-4e3d-8466-064bda6e780b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.253326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefb9fc-8e45-4e3d-8466-064bda6e780b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:46.255247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefb9fc-8e45-4e3d-8466-064bda6e780b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:46.266392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1000/2018] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:46.271208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bdfc2-7f38-4f78-b545-6e3b513bc756 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:46.281937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bdfc2-7f38-4f78-b545-6e3b513bc756 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:46.288111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2a242-0ef3-440e-a452-cc24a0642d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:46.289037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2a242-0ef3-440e-a452-cc24a0642d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.289320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2a242-0ef3-440e-a452-cc24a0642d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.289709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2a242-0ef3-440e-a452-cc24a0642d82 admin admin] PATH_INFO: `/v3/projects/dad0e3e647144a029f8bdf2194d09681/users/5a24cb7d0b114958a31857b06bccba98/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.306939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2a242-0ef3-440e-a452-cc24a0642d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dad0e3e647144a029f8bdf2194d09681, user_id=5a24cb7d0b114958a31857b06bccba98, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:46.309595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2a242-0ef3-440e-a452-cc24a0642d82 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:46.327534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1019/2019] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:46 2022] PUT /identity/v3/projects/dad0e3e647144a029f8bdf2194d09681/users/5a24cb7d0b114958a31857b06bccba98/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:36:46.338841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb258bc-5e1d-4c27-afda-e7624d3a076f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.339128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb258bc-5e1d-4c27-afda-e7624d3a076f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.339406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb258bc-5e1d-4c27-afda-e7624d3a076f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.360100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:46.368419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9eb258bc-5e1d-4c27-afda-e7624d3a076f None None] MFA Rules not processed for user `5a24cb7d0b114958a31857b06bccba98`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:46.370272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9eb258bc-5e1d-4c27-afda-e7624d3a076f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:46.434203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1001/2020] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:46.439119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2a242-0ef3-440e-a452-cc24a0642d82 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:46.505436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2a242-0ef3-440e-a452-cc24a0642d82 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:46.510114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b019ed5-4eda-4656-8892-5eb222ba3a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:46.510809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b019ed5-4eda-4656-8892-5eb222ba3a5d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.511094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b019ed5-4eda-4656-8892-5eb222ba3a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.511371 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b019ed5-4eda-4656-8892-5eb222ba3a5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.512073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b019ed5-4eda-4656-8892-5eb222ba3a5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:46.513995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b019ed5-4eda-4656-8892-5eb222ba3a5d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:46.539914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1020/2021] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:36:46 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:36:46.543940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb258bc-5e1d-4c27-afda-e7624d3a076f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:46.555008 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb258bc-5e1d-4c27-afda-e7624d3a076f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:46.611431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0e251-01bd-4492-99d5-1970bd8dbaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:46.612023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0e251-01bd-4492-99d5-1970bd8dbaf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.612305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0e251-01bd-4492-99d5-1970bd8dbaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.612580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0e251-01bd-4492-99d5-1970bd8dbaf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.613268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0e251-01bd-4492-99d5-1970bd8dbaf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:46.615208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea0e251-01bd-4492-99d5-1970bd8dbaf7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:46.636825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1002/2022] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:36:46 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:36:46.641572 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b019ed5-4eda-4656-8892-5eb222ba3a5d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:46.652082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b019ed5-4eda-4656-8892-5eb222ba3a5d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:46.656963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2412b-6803-4da3-a8e9-1d920a6dba16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:46.657616 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2412b-6803-4da3-a8e9-1d920a6dba16 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.657893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2412b-6803-4da3-a8e9-1d920a6dba16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.658168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2412b-6803-4da3-a8e9-1d920a6dba16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.658910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2412b-6803-4da3-a8e9-1d920a6dba16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:46.660501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2412b-6803-4da3-a8e9-1d920a6dba16 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:46.671711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1021/2023] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:46.676205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0e251-01bd-4492-99d5-1970bd8dbaf7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:46.687568 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0e251-01bd-4492-99d5-1970bd8dbaf7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:46.696165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a8b39-0ea7-46e4-b998-f318cb46c011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:46.705307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a8b39-0ea7-46e4-b998-f318cb46c011 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.705784 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a8b39-0ea7-46e4-b998-f318cb46c011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.706129 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a8b39-0ea7-46e4-b998-f318cb46c011 admin admin] PATH_INFO: `/v3/projects/40fdd9dfb2ff4c5a82cba3340df16280/users/488f1f80315f467fae39ee319af6d426/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.728747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a8b39-0ea7-46e4-b998-f318cb46c011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fdd9dfb2ff4c5a82cba3340df16280, user_id=488f1f80315f467fae39ee319af6d426, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:46.731811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a8b39-0ea7-46e4-b998-f318cb46c011 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:46.747665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1003/2024] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:46 2022] PUT /identity/v3/projects/40fdd9dfb2ff4c5a82cba3340df16280/users/488f1f80315f467fae39ee319af6d426/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:46.753316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2412b-6803-4da3-a8e9-1d920a6dba16 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:46.768635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2412b-6803-4da3-a8e9-1d920a6dba16 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:46.812433 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:46.844641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc5cb3-f7a7-4671-ae27-c6bb6522f5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:46.845323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc5cb3-f7a7-4671-ae27-c6bb6522f5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.845760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc5cb3-f7a7-4671-ae27-c6bb6522f5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.846036 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc5cb3-f7a7-4671-ae27-c6bb6522f5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.846827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc5cb3-f7a7-4671-ae27-c6bb6522f5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:46.848500 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc5cb3-f7a7-4671-ae27-c6bb6522f5d0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:46.859687 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1022/2025] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:36:46.863881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a8b39-0ea7-46e4-b998-f318cb46c011 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:46.874609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a8b39-0ea7-46e4-b998-f318cb46c011 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:46.879683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d28f4-cd3d-4e7b-beb2-6c5a51956438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:46.880671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d28f4-cd3d-4e7b-beb2-6c5a51956438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.881009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d28f4-cd3d-4e7b-beb2-6c5a51956438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.881287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d28f4-cd3d-4e7b-beb2-6c5a51956438 admin admin] PATH_INFO: `/v3/projects/40fdd9dfb2ff4c5a82cba3340df16280/users/488f1f80315f467fae39ee319af6d426/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.883425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d28f4-cd3d-4e7b-beb2-6c5a51956438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40fdd9dfb2ff4c5a82cba3340df16280, user_id=488f1f80315f467fae39ee319af6d426, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:46.886063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d28f4-cd3d-4e7b-beb2-6c5a51956438 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:46.897596 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1004/2026] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:36:46 2022] PUT /identity/v3/projects/40fdd9dfb2ff4c5a82cba3340df16280/users/488f1f80315f467fae39ee319af6d426/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:36:46.907930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31389853-a5f5-43c2-aa0f-235e192cd298 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:46.908355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31389853-a5f5-43c2-aa0f-235e192cd298 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:46.908634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31389853-a5f5-43c2-aa0f-235e192cd298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:46.936892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-31389853-a5f5-43c2-aa0f-235e192cd298 None None] MFA Rules not processed for user `488f1f80315f467fae39ee319af6d426`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:36:46.939783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-31389853-a5f5-43c2-aa0f-235e192cd298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:47.012671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1023/2027] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:36:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:36:47.028066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d28f4-cd3d-4e7b-beb2-6c5a51956438 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:47.029887 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-889d28f4-cd3d-4e7b-beb2-6c5a51956438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:47.096013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d28f4-cd3d-4e7b-beb2-6c5a51956438 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:47.099832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97ce4b-4954-41c5-9c17-1ea40bbd4148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:47.100536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97ce4b-4954-41c5-9c17-1ea40bbd4148 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:47.100818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97ce4b-4954-41c5-9c17-1ea40bbd4148 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:47.101096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97ce4b-4954-41c5-9c17-1ea40bbd4148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:47.108370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:47.111141 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97ce4b-4954-41c5-9c17-1ea40bbd4148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:47.112970 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97ce4b-4954-41c5-9c17-1ea40bbd4148 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:47.123788 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1005/2028] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:47.163845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31389853-a5f5-43c2-aa0f-235e192cd298 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:36:47.165397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-31389853-a5f5-43c2-aa0f-235e192cd298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:36:47.180036 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31389853-a5f5-43c2-aa0f-235e192cd298 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:36:47.184760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aefaa5-0712-4e7d-b0aa-b24dff5e9d4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:36:47.185707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aefaa5-0712-4e7d-b0aa-b24dff5e9d4f service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:36:47.186086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aefaa5-0712-4e7d-b0aa-b24dff5e9d4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:36:47.186425 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aefaa5-0712-4e7d-b0aa-b24dff5e9d4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:36:47.245823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aefaa5-0712-4e7d-b0aa-b24dff5e9d4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:36:47.247319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aefaa5-0712-4e7d-b0aa-b24dff5e9d4f service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:36:47.256365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1024/2029] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:36:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:36:57.122776 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:36:57.252345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=71 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:01.880451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97ce4b-4954-41c5-9c17-1ea40bbd4148 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:01.950183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97ce4b-4954-41c5-9c17-1ea40bbd4148 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:01.954900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:01.955786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:01.956081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:01.956358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] PATH_INFO: `/v3/users/4802ea8a54f64f53ae4b18bded752836` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:01.957636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4802ea8a54f64f53ae4b18bded752836)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:01.959637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:02.011412 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4802ea8a54f64f53ae4b18bded752836'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.012066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4802ea8a54f64f53ae4b18bded752836'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.028572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4802ea8a54f64f53ae4b18bded752836'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.029321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4802ea8a54f64f53ae4b18bded752836'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.039840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4802ea8a54f64f53ae4b18bded752836'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.060838 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1006/2030] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:01 2022] DELETE /identity/v3/users/4802ea8a54f64f53ae4b18bded752836 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:02.062091 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730d6d5-8412-4c63-af70-12982cbff6ca None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:02.062539 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730d6d5-8412-4c63-af70-12982cbff6ca None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:02.062615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730d6d5-8412-4c63-af70-12982cbff6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:02.081611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8730d6d5-8412-4c63-af70-12982cbff6ca None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:02.082880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8730d6d5-8412-4c63-af70-12982cbff6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:02.166095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1025/2031] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:02.175675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:02.177105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:02.243346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb642ffb-1bb4-4085-9174-e7702b35ce61 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:02.247921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b755fc1-891d-49fe-bf67-c1f9aed00f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:02.249041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b755fc1-891d-49fe-bf67-c1f9aed00f7e service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:02.249345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b755fc1-891d-49fe-bf67-c1f9aed00f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:02.249647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b755fc1-891d-49fe-bf67-c1f9aed00f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:02.266116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b755fc1-891d-49fe-bf67-c1f9aed00f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:02.268230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b755fc1-891d-49fe-bf67-c1f9aed00f7e service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:02.282636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1007/2032] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:02.608595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730d6d5-8412-4c63-af70-12982cbff6ca None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:02.610250 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8730d6d5-8412-4c63-af70-12982cbff6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:02.621874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b755fc1-891d-49fe-bf67-c1f9aed00f7e service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:02.623429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0b755fc1-891d-49fe-bf67-c1f9aed00f7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:02.626979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730d6d5-8412-4c63-af70-12982cbff6ca None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:02.631825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:02.632789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:02.633223 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:02.633542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] PATH_INFO: `/v3/projects/9bf344fd1761412a9c4601592b43cd0c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:02.635151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf344fd1761412a9c4601592b43cd0c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:02.639880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:02.649763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b755fc1-891d-49fe-bf67-c1f9aed00f7e service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:02.665655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b621ce-5222-4809-a0e9-a0504f398b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:02.666571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b621ce-5222-4809-a0e9-a0504f398b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:02.666882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b621ce-5222-4809-a0e9-a0504f398b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:02.667159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b621ce-5222-4809-a0e9-a0504f398b91 admin admin] PATH_INFO: `/v3/users/3958ea727248421ca8277aaa29fa10e4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:02.668382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b621ce-5222-4809-a0e9-a0504f398b91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3958ea727248421ca8277aaa29fa10e4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:02.670672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b621ce-5222-4809-a0e9-a0504f398b91 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:02.698005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:02.717933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf344fd1761412a9c4601592b43cd0c', 'request_id': 'req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.732051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf344fd1761412a9c4601592b43cd0c', 'request_id': 'req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.740839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf344fd1761412a9c4601592b43cd0c', 'request_id': 'req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.754252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c1b621ce-5222-4809-a0e9-a0504f398b91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3958ea727248421ca8277aaa29fa10e4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.755048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c1b621ce-5222-4809-a0e9-a0504f398b91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3958ea727248421ca8277aaa29fa10e4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.757858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] The token cache is being invalidate because project 9bf344fd1761412a9c4601592b43cd0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:02.758145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.758850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.760792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1026/2033] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:02 2022] DELETE /identity/v3/projects/9bf344fd1761412a9c4601592b43cd0c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:02.765515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:02.769026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:02.773046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c1b621ce-5222-4809-a0e9-a0504f398b91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3958ea727248421ca8277aaa29fa10e4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.773846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c1b621ce-5222-4809-a0e9-a0504f398b91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3958ea727248421ca8277aaa29fa10e4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.784597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c1b621ce-5222-4809-a0e9-a0504f398b91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3958ea727248421ca8277aaa29fa10e4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.802428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1008/2034] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:02 2022] DELETE /identity/v3/users/3958ea727248421ca8277aaa29fa10e4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:02.806248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb55e-d94e-4f64-8f0b-091743aa7322 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:02.806539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb55e-d94e-4f64-8f0b-091743aa7322 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:02.806858 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cb55e-d94e-4f64-8f0b-091743aa7322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:02.826005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4b6cb55e-d94e-4f64-8f0b-091743aa7322 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:02.827978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4b6cb55e-d94e-4f64-8f0b-091743aa7322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:02.846793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933e4b8-ff2f-4d9d-82dd-7db71a9e7bee admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:02.880726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:02.902851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1009/2035] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:02.914149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cb55e-d94e-4f64-8f0b-091743aa7322 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:02.915799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4b6cb55e-d94e-4f64-8f0b-091743aa7322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:02.916112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:02.917026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:02.917304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:02.917641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] PATH_INFO: `/v3/users/5a24cb7d0b114958a31857b06bccba98` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:02.919115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a24cb7d0b114958a31857b06bccba98)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:02.921027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:02.995582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a24cb7d0b114958a31857b06bccba98'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:02.996629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a24cb7d0b114958a31857b06bccba98'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:03.000144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cb55e-d94e-4f64-8f0b-091743aa7322 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:03.005967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb232c4-7a51-4efa-a26f-cca6234a10f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:03.006982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb232c4-7a51-4efa-a26f-cca6234a10f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:03.008010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb232c4-7a51-4efa-a26f-cca6234a10f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:03.008365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb232c4-7a51-4efa-a26f-cca6234a10f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:03.011215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a24cb7d0b114958a31857b06bccba98'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:03.011364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3fb232c4-7a51-4efa-a26f-cca6234a10f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:03.011928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a24cb7d0b114958a31857b06bccba98'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:03.022530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a24cb7d0b114958a31857b06bccba98'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:03.027489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb232c4-7a51-4efa-a26f-cca6234a10f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:03.029331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb232c4-7a51-4efa-a26f-cca6234a10f4 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:03.037367 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1027/2036] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:02 2022] DELETE /identity/v3/users/5a24cb7d0b114958a31857b06bccba98 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:03.040445 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1010/2037] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:37:03.836373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:03.838232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:03.893413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb232c4-7a51-4efa-a26f-cca6234a10f4 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:03.895653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3fb232c4-7a51-4efa-a26f-cca6234a10f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:03.912545 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb232c4-7a51-4efa-a26f-cca6234a10f4 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:03.941499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a03837-e443-430a-85ae-db8b49b251c7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:03.950150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:03.951081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:03.951363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:03.951637 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] PATH_INFO: `/v3/projects/02f8382872514195b5d03b46775f7485` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:03.953502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02f8382872514195b5d03b46775f7485)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:03.957280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:03.978933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:04.028594 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:04.029663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.030011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.030349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] PATH_INFO: `/v3/projects/dad0e3e647144a029f8bdf2194d09681` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.030661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02f8382872514195b5d03b46775f7485', 'request_id': 'req-49a5349b-b8cc-46a1-bb93-385cf2eaed50'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.032850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:04.033539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dad0e3e647144a029f8bdf2194d09681)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:04.036868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:04.040603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02f8382872514195b5d03b46775f7485', 'request_id': 'req-49a5349b-b8cc-46a1-bb93-385cf2eaed50'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.048579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02f8382872514195b5d03b46775f7485', 'request_id': 'req-49a5349b-b8cc-46a1-bb93-385cf2eaed50'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.071543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] The token cache is being invalidate because project 02f8382872514195b5d03b46775f7485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:04.071838 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.072507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49a5349b-b8cc-46a1-bb93-385cf2eaed50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.074267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1028/2038] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:03 2022] DELETE /identity/v3/projects/02f8382872514195b5d03b46775f7485 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:04.078757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669a5c2-d008-4539-ae1a-380580066e5b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.079056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669a5c2-d008-4539-ae1a-380580066e5b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.079342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669a5c2-d008-4539-ae1a-380580066e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.098973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5669a5c2-d008-4539-ae1a-380580066e5b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:04.100579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5669a5c2-d008-4539-ae1a-380580066e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:04.106185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dad0e3e647144a029f8bdf2194d09681', 'request_id': 'req-a290c31c-89db-4160-b707-18a5dd4cc358'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.113461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dad0e3e647144a029f8bdf2194d09681', 'request_id': 'req-a290c31c-89db-4160-b707-18a5dd4cc358'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.120345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dad0e3e647144a029f8bdf2194d09681', 'request_id': 'req-a290c31c-89db-4160-b707-18a5dd4cc358'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.137013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] The token cache is being invalidate because project dad0e3e647144a029f8bdf2194d09681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:04.137543 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.138515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.143273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1011/2039] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:03 2022] DELETE /identity/v3/projects/dad0e3e647144a029f8bdf2194d09681 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:04.144665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:04.146300 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:04.166610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1029/2040] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:37:04.174542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589d2c5-e7ba-418d-95ae-105ad8edcc75 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.174875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589d2c5-e7ba-418d-95ae-105ad8edcc75 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.175160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b589d2c5-e7ba-418d-95ae-105ad8edcc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.194983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b589d2c5-e7ba-418d-95ae-105ad8edcc75 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:04.196208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b589d2c5-e7ba-418d-95ae-105ad8edcc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:04.210874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1030/2041] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:04.215373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589d2c5-e7ba-418d-95ae-105ad8edcc75 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:04.220979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290c31c-89db-4160-b707-18a5dd4cc358 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:04.225551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b589d2c5-e7ba-418d-95ae-105ad8edcc75 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:04.227161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:04.230184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.230184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.230184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] PATH_INFO: `/v3/users/488f1f80315f467fae39ee319af6d426` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.230184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=488f1f80315f467fae39ee319af6d426)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:04.231916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:04.233549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8efb5-3232-4e18-a4d0-28c65e88b55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:04.234523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8efb5-3232-4e18-a4d0-28c65e88b55b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.234878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8efb5-3232-4e18-a4d0-28c65e88b55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.235189 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8efb5-3232-4e18-a4d0-28c65e88b55b admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.236152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8efb5-3232-4e18-a4d0-28c65e88b55b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:37:04.236467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8efb5-3232-4e18-a4d0-28c65e88b55b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:04.238292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8efb5-3232-4e18-a4d0-28c65e88b55b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:04.257647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1031/2042] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:37:04.261769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8efb5-3232-4e18-a4d0-28c65e88b55b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:04.274125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8efb5-3232-4e18-a4d0-28c65e88b55b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:04.279859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7b854-4aa4-46a9-ae5e-4e20fba15e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:04.280533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7b854-4aa4-46a9-ae5e-4e20fba15e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.280821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7b854-4aa4-46a9-ae5e-4e20fba15e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.281481 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7b854-4aa4-46a9-ae5e-4e20fba15e1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.281863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7b854-4aa4-46a9-ae5e-4e20fba15e1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:04.283854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7b854-4aa4-46a9-ae5e-4e20fba15e1d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:04.293455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '488f1f80315f467fae39ee319af6d426'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.294685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '488f1f80315f467fae39ee319af6d426'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.310547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '488f1f80315f467fae39ee319af6d426'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.313056 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '488f1f80315f467fae39ee319af6d426'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.318863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:04.321945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1032/2043] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:37:04 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:37:04.322871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7b854-4aa4-46a9-ae5e-4e20fba15e1d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:04.324866 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '488f1f80315f467fae39ee319af6d426'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:04.325146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8df7b854-4aa4-46a9-ae5e-4e20fba15e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:04.347085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1012/2044] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:04 2022] DELETE /identity/v3/users/488f1f80315f467fae39ee319af6d426 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:04.408484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7b854-4aa4-46a9-ae5e-4e20fba15e1d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:04.464398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:04.480889 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed1db3-4619-4114-b896-0e39cacb046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:04.481591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed1db3-4619-4114-b896-0e39cacb046c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.482170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed1db3-4619-4114-b896-0e39cacb046c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.482657 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed1db3-4619-4114-b896-0e39cacb046c admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.483725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed1db3-4619-4114-b896-0e39cacb046c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:04.485734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed1db3-4619-4114-b896-0e39cacb046c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:04.509531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1033/2045] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:37:04 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:04.513889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:04.528595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58203aa1-d640-4cce-b367-575d4b405a5d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:04.533875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3681b7-2d61-408e-8c31-f01d0bf88f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:04.534606 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3681b7-2d61-408e-8c31-f01d0bf88f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.534923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3681b7-2d61-408e-8c31-f01d0bf88f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.535309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3681b7-2d61-408e-8c31-f01d0bf88f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.536300 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3681b7-2d61-408e-8c31-f01d0bf88f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:04.537732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3681b7-2d61-408e-8c31-f01d0bf88f41 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:04.553607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1013/2046] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:04.558262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed1db3-4619-4114-b896-0e39cacb046c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:04.572819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed1db3-4619-4114-b896-0e39cacb046c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:04.583630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9eed5-b2f0-4629-aca9-528f024f51e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:04.584877 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9eed5-b2f0-4629-aca9-528f024f51e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.585230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9eed5-b2f0-4629-aca9-528f024f51e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.585571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9eed5-b2f0-4629-aca9-528f024f51e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.588164 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9eed5-b2f0-4629-aca9-528f024f51e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:04.592167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9eed5-b2f0-4629-aca9-528f024f51e3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:04.617595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1034/2047] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:37:04.621803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3681b7-2d61-408e-8c31-f01d0bf88f41 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:04.633940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3681b7-2d61-408e-8c31-f01d0bf88f41 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:04.640053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a0196-b534-4f10-a1ae-e4ac37addcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:04.641237 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a0196-b534-4f10-a1ae-e4ac37addcf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.642148 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a0196-b534-4f10-a1ae-e4ac37addcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.642523 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a0196-b534-4f10-a1ae-e4ac37addcf6 admin admin] PATH_INFO: `/v3/projects/3faae8ed99784a52b784c22e77de9890/users/ac61282884a54556bb611c59c30d75a2/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.660329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a0196-b534-4f10-a1ae-e4ac37addcf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3faae8ed99784a52b784c22e77de9890, user_id=ac61282884a54556bb611c59c30d75a2, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:04.663330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68a0196-b534-4f10-a1ae-e4ac37addcf6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:04.694979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066c157-6371-4443-953e-87be5a84258d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.695374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066c157-6371-4443-953e-87be5a84258d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.695728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066c157-6371-4443-953e-87be5a84258d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.697998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1014/2048] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:04 2022] PUT /identity/v3/projects/3faae8ed99784a52b784c22e77de9890/users/ac61282884a54556bb611c59c30d75a2/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:04.737945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3066c157-6371-4443-953e-87be5a84258d None None] MFA Rules not processed for user `ac61282884a54556bb611c59c30d75a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:04.739579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3066c157-6371-4443-953e-87be5a84258d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:04.851531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a0196-b534-4f10-a1ae-e4ac37addcf6 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:04.853574 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e68a0196-b534-4f10-a1ae-e4ac37addcf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:04.856278 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1035/2049] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:04.883157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066c157-6371-4443-953e-87be5a84258d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:04.885254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3066c157-6371-4443-953e-87be5a84258d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:04.979305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066c157-6371-4443-953e-87be5a84258d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:04.988172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a0196-b534-4f10-a1ae-e4ac37addcf6 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:04.990145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f565d426-6183-464b-a436-8fd60a996d37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:04.990859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565d426-6183-464b-a436-8fd60a996d37 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.991142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565d426-6183-464b-a436-8fd60a996d37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.991418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f565d426-6183-464b-a436-8fd60a996d37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.994205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:04.995934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:04.996175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:04.996461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] PATH_INFO: `/v3/projects/40fdd9dfb2ff4c5a82cba3340df16280` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:04.998196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40fdd9dfb2ff4c5a82cba3340df16280)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:05.000658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:05.012662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565d426-6183-464b-a436-8fd60a996d37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:05.015433 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f565d426-6183-464b-a436-8fd60a996d37 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:05.030434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1036/2050] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:05.080860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40fdd9dfb2ff4c5a82cba3340df16280', 'request_id': 'req-7417335c-1019-4eb2-bae4-0e547c7c8874'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:05.088352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40fdd9dfb2ff4c5a82cba3340df16280', 'request_id': 'req-7417335c-1019-4eb2-bae4-0e547c7c8874'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:05.114589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40fdd9dfb2ff4c5a82cba3340df16280', 'request_id': 'req-7417335c-1019-4eb2-bae4-0e547c7c8874'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:05.146657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] The token cache is being invalidate because project 40fdd9dfb2ff4c5a82cba3340df16280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:05.146974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:05.148389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7417335c-1019-4eb2-bae4-0e547c7c8874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:05.153054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1015/2051] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:04 2022] DELETE /identity/v3/projects/40fdd9dfb2ff4c5a82cba3340df16280 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:05.159254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea8b7e-06b8-4246-b785-410122fdefae None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:05.159850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea8b7e-06b8-4246-b785-410122fdefae None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:05.160326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dea8b7e-06b8-4246-b785-410122fdefae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:05.190042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5dea8b7e-06b8-4246-b785-410122fdefae None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:05.191928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5dea8b7e-06b8-4246-b785-410122fdefae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:05.265337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:05.284936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1037/2052] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:05.295821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd078c4-8096-40ac-b395-d3b41e2ceafd None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:05.296408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd078c4-8096-40ac-b395-d3b41e2ceafd None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:05.296837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd078c4-8096-40ac-b395-d3b41e2ceafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:05.323604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7bd078c4-8096-40ac-b395-d3b41e2ceafd None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:05.325471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7bd078c4-8096-40ac-b395-d3b41e2ceafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:05.337828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1016/2053] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:05.343075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea8b7e-06b8-4246-b785-410122fdefae None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:05.356132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dea8b7e-06b8-4246-b785-410122fdefae None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:05.363257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff242c-796f-4bf1-889a-0e3a961a1b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:05.364188 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff242c-796f-4bf1-889a-0e3a961a1b5a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:05.364512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff242c-796f-4bf1-889a-0e3a961a1b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:05.365097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff242c-796f-4bf1-889a-0e3a961a1b5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:05.366133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff242c-796f-4bf1-889a-0e3a961a1b5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:37:05.366450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff242c-796f-4bf1-889a-0e3a961a1b5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:05.368375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff242c-796f-4bf1-889a-0e3a961a1b5a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:05.394994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1038/2054] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:37:05.401383 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd078c4-8096-40ac-b395-d3b41e2ceafd None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:05.419713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd078c4-8096-40ac-b395-d3b41e2ceafd None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:05.425294 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73843eb4-4f16-4c00-a1b2-f4659f8fe489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:05.426064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73843eb4-4f16-4c00-a1b2-f4659f8fe489 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:05.426346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73843eb4-4f16-4c00-a1b2-f4659f8fe489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:05.426651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73843eb4-4f16-4c00-a1b2-f4659f8fe489 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:05.427679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73843eb4-4f16-4c00-a1b2-f4659f8fe489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:05.429644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73843eb4-4f16-4c00-a1b2-f4659f8fe489 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:05.528872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1017/2055] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:37:05 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:05.533332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff242c-796f-4bf1-889a-0e3a961a1b5a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:05.547167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff242c-796f-4bf1-889a-0e3a961a1b5a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:05.629437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeed013-4b74-4c39-b879-45ae617d6445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:05.630149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeed013-4b74-4c39-b879-45ae617d6445 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:05.630558 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeed013-4b74-4c39-b879-45ae617d6445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:05.630830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeed013-4b74-4c39-b879-45ae617d6445 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:05.631616 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeed013-4b74-4c39-b879-45ae617d6445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:05.633694 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeed013-4b74-4c39-b879-45ae617d6445 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:05.657150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1039/2056] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:37:05 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:05.662787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73843eb4-4f16-4c00-a1b2-f4659f8fe489 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:05.674819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73843eb4-4f16-4c00-a1b2-f4659f8fe489 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:05.680796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1fc82-3319-4aa5-bdf4-91e04c9e1840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:05.681503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1fc82-3319-4aa5-bdf4-91e04c9e1840 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:05.681783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1fc82-3319-4aa5-bdf4-91e04c9e1840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:05.682072 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1fc82-3319-4aa5-bdf4-91e04c9e1840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:05.682855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1fc82-3319-4aa5-bdf4-91e04c9e1840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:05.684520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb1fc82-3319-4aa5-bdf4-91e04c9e1840 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:05.697184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1018/2057] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:05.701737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeed013-4b74-4c39-b879-45ae617d6445 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:05.712650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeed013-4b74-4c39-b879-45ae617d6445 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:05.717980 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4b338-9d4e-4b14-9b58-dc71294f5748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:05.718787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4b338-9d4e-4b14-9b58-dc71294f5748 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:05.719007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4b338-9d4e-4b14-9b58-dc71294f5748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:05.719279 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4b338-9d4e-4b14-9b58-dc71294f5748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:05.720021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4b338-9d4e-4b14-9b58-dc71294f5748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:05.721628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4b338-9d4e-4b14-9b58-dc71294f5748 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:05.733424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1040/2058] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:37:05.738037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1fc82-3319-4aa5-bdf4-91e04c9e1840 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:05.743992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:05.749932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1fc82-3319-4aa5-bdf4-91e04c9e1840 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:05.756455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b35e57-ad3f-4688-96cc-40b05baff6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:05.757562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b35e57-ad3f-4688-96cc-40b05baff6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:05.758718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b35e57-ad3f-4688-96cc-40b05baff6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:05.759127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b35e57-ad3f-4688-96cc-40b05baff6e7 admin admin] PATH_INFO: `/v3/projects/814419d346854a7fb081b467a3a8fe9a/users/9fb54d1028f94f639587fdc775dbe9f5/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:05.774876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b35e57-ad3f-4688-96cc-40b05baff6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=814419d346854a7fb081b467a3a8fe9a, user_id=9fb54d1028f94f639587fdc775dbe9f5, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:05.777733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b35e57-ad3f-4688-96cc-40b05baff6e7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:05.792079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1019/2059] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:05 2022] PUT /identity/v3/projects/814419d346854a7fb081b467a3a8fe9a/users/9fb54d1028f94f639587fdc775dbe9f5/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:05.804208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3283912-ed3c-489b-9c2a-2a5fa31ecd3b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:05.804499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3283912-ed3c-489b-9c2a-2a5fa31ecd3b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:05.804787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3283912-ed3c-489b-9c2a-2a5fa31ecd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:05.835096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a3283912-ed3c-489b-9c2a-2a5fa31ecd3b None None] MFA Rules not processed for user `9fb54d1028f94f639587fdc775dbe9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:05.836360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a3283912-ed3c-489b-9c2a-2a5fa31ecd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:05.911354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1041/2060] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:05.916422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b35e57-ad3f-4688-96cc-40b05baff6e7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:05.994900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b35e57-ad3f-4688-96cc-40b05baff6e7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:06.000506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9e658-fdcc-4400-a8d4-9c9e204a550e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:06.001248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9e658-fdcc-4400-a8d4-9c9e204a550e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:06.001637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9e658-fdcc-4400-a8d4-9c9e204a550e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:06.002011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d9e658-fdcc-4400-a8d4-9c9e204a550e admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:06.002984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9e658-fdcc-4400-a8d4-9c9e204a550e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:06.011626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d9e658-fdcc-4400-a8d4-9c9e204a550e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:06.048378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1020/2061] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:37:05 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:06.052998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3283912-ed3c-489b-9c2a-2a5fa31ecd3b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:06.065026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3283912-ed3c-489b-9c2a-2a5fa31ecd3b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:06.117531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:06.127531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ba704-dbf7-49ab-bf65-c38cfa060be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:06.128215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ba704-dbf7-49ab-bf65-c38cfa060be1 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:06.128497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ba704-dbf7-49ab-bf65-c38cfa060be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:06.128775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ba704-dbf7-49ab-bf65-c38cfa060be1 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:06.129477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ba704-dbf7-49ab-bf65-c38cfa060be1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:06.132721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50ba704-dbf7-49ab-bf65-c38cfa060be1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:06.156943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1042/2062] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:37:06 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:06.160728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9e658-fdcc-4400-a8d4-9c9e204a550e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:06.171291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d9e658-fdcc-4400-a8d4-9c9e204a550e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:06.176018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26bd93-ce42-440d-831a-508c075717b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:06.176692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26bd93-ce42-440d-831a-508c075717b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:06.176978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26bd93-ce42-440d-831a-508c075717b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:06.177376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26bd93-ce42-440d-831a-508c075717b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:06.178116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26bd93-ce42-440d-831a-508c075717b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:06.179762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26bd93-ce42-440d-831a-508c075717b5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:06.190652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1021/2063] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:06.195174 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ba704-dbf7-49ab-bf65-c38cfa060be1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:06.205567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ba704-dbf7-49ab-bf65-c38cfa060be1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:06.211359 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a54a97-66d9-4adb-8242-ec83cdc4be9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:06.212829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54a97-66d9-4adb-8242-ec83cdc4be9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:06.212829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54a97-66d9-4adb-8242-ec83cdc4be9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:06.213004 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a54a97-66d9-4adb-8242-ec83cdc4be9a admin admin] PATH_INFO: `/v3/projects/c04facab84d54404842657efd8192186/users/c6c7a82b6f714a18be96f69cf1320ce3/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:06.240055 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a54a97-66d9-4adb-8242-ec83cdc4be9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04facab84d54404842657efd8192186, user_id=c6c7a82b6f714a18be96f69cf1320ce3, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:06.242798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a54a97-66d9-4adb-8242-ec83cdc4be9a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:06.270009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1043/2064] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:06 2022] PUT /identity/v3/projects/c04facab84d54404842657efd8192186/users/c6c7a82b6f714a18be96f69cf1320ce3/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:06.274262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26bd93-ce42-440d-831a-508c075717b5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:06.284503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26bd93-ce42-440d-831a-508c075717b5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:06.347100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d14406-d542-4a67-88cf-923d7759448d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:06.347639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d14406-d542-4a67-88cf-923d7759448d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:06.347924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d14406-d542-4a67-88cf-923d7759448d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:06.348199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d14406-d542-4a67-88cf-923d7759448d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:06.348945 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d14406-d542-4a67-88cf-923d7759448d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:06.350839 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d14406-d542-4a67-88cf-923d7759448d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:06.361691 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1022/2065] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:37:06.366171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a54a97-66d9-4adb-8242-ec83cdc4be9a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:06.376943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a54a97-66d9-4adb-8242-ec83cdc4be9a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:06.381844 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a839fa-f4cc-4373-b95c-4e618f5d3fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:06.382773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a839fa-f4cc-4373-b95c-4e618f5d3fef admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:06.383057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a839fa-f4cc-4373-b95c-4e618f5d3fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:06.383340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a839fa-f4cc-4373-b95c-4e618f5d3fef admin admin] PATH_INFO: `/v3/projects/c04facab84d54404842657efd8192186/users/c6c7a82b6f714a18be96f69cf1320ce3/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:06.385419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a839fa-f4cc-4373-b95c-4e618f5d3fef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c04facab84d54404842657efd8192186, user_id=c6c7a82b6f714a18be96f69cf1320ce3, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:06.388179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a839fa-f4cc-4373-b95c-4e618f5d3fef admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:06.411121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1044/2066] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:06 2022] PUT /identity/v3/projects/c04facab84d54404842657efd8192186/users/c6c7a82b6f714a18be96f69cf1320ce3/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:06.414122 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c30fdb-08a6-4b86-a7fd-07792f55aa2a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:06.414439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c30fdb-08a6-4b86-a7fd-07792f55aa2a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:06.414756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c30fdb-08a6-4b86-a7fd-07792f55aa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:06.447451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-54c30fdb-08a6-4b86-a7fd-07792f55aa2a None None] MFA Rules not processed for user `c6c7a82b6f714a18be96f69cf1320ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:06.449082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-54c30fdb-08a6-4b86-a7fd-07792f55aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:06.498330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:06.532376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1023/2067] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:06.543154 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a839fa-f4cc-4373-b95c-4e618f5d3fef admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:06.544861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f5a839fa-f4cc-4373-b95c-4e618f5d3fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:06.614319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a839fa-f4cc-4373-b95c-4e618f5d3fef admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:06.618476 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51162caa-742f-4673-ba30-3fe69bbcdc58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:06.619156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51162caa-742f-4673-ba30-3fe69bbcdc58 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:06.619438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51162caa-742f-4673-ba30-3fe69bbcdc58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:06.619713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51162caa-742f-4673-ba30-3fe69bbcdc58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:06.629410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51162caa-742f-4673-ba30-3fe69bbcdc58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:06.631020 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51162caa-742f-4673-ba30-3fe69bbcdc58 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:06.640609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1045/2068] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:07.250407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c30fdb-08a6-4b86-a7fd-07792f55aa2a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:07.319348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c30fdb-08a6-4b86-a7fd-07792f55aa2a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:07.323900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:07.324804 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:07.325092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:07.325370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] PATH_INFO: `/v3/users/9fb54d1028f94f639587fdc775dbe9f5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:07.326610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb54d1028f94f639587fdc775dbe9f5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:07.328559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:07.385873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb54d1028f94f639587fdc775dbe9f5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:07.386465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb54d1028f94f639587fdc775dbe9f5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:07.398931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb54d1028f94f639587fdc775dbe9f5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:07.399561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb54d1028f94f639587fdc775dbe9f5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:07.409117 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb54d1028f94f639587fdc775dbe9f5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:07.421691 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1024/2069] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:07 2022] DELETE /identity/v3/users/9fb54d1028f94f639587fdc775dbe9f5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:07.426361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06632c49-ff5e-4797-b1d1-f4bba85c0e9a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:07.426905 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06632c49-ff5e-4797-b1d1-f4bba85c0e9a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:07.427015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06632c49-ff5e-4797-b1d1-f4bba85c0e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:07.446163 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-06632c49-ff5e-4797-b1d1-f4bba85c0e9a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:07.447410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-06632c49-ff5e-4797-b1d1-f4bba85c0e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:07.516597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1046/2070] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:07.526211 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:07.527741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:07.590712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3441b4f-64dd-43d3-8179-03a8376e0305 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:07.594799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8144ae8-5f13-4cf2-8bf3-0e304bf87384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:07.595456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8144ae8-5f13-4cf2-8bf3-0e304bf87384 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:07.595743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8144ae8-5f13-4cf2-8bf3-0e304bf87384 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:07.596014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8144ae8-5f13-4cf2-8bf3-0e304bf87384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:07.604075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:07.606643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8144ae8-5f13-4cf2-8bf3-0e304bf87384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:07.608446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8144ae8-5f13-4cf2-8bf3-0e304bf87384 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:07.622974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1025/2071] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:07.634546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562e0a7-a44e-485d-a857-7283ce9a53fa None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:07.634881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562e0a7-a44e-485d-a857-7283ce9a53fa None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:07.635167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562e0a7-a44e-485d-a857-7283ce9a53fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:07.654646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2562e0a7-a44e-485d-a857-7283ce9a53fa None None] MFA Rules not processed for user `ac61282884a54556bb611c59c30d75a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:07.655827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2562e0a7-a44e-485d-a857-7283ce9a53fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:07.702991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:07.717131 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1047/2072] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:08.089258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8144ae8-5f13-4cf2-8bf3-0e304bf87384 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:08.091334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d8144ae8-5f13-4cf2-8bf3-0e304bf87384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:08.108911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8144ae8-5f13-4cf2-8bf3-0e304bf87384 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:08.114079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:08.114999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:08.115359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:08.115643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] PATH_INFO: `/v3/projects/814419d346854a7fb081b467a3a8fe9a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:08.117071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=814419d346854a7fb081b467a3a8fe9a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:08.119156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:08.175783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '814419d346854a7fb081b467a3a8fe9a', 'request_id': 'req-d230c612-770c-4c73-a61c-a82933c686bd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.182422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '814419d346854a7fb081b467a3a8fe9a', 'request_id': 'req-d230c612-770c-4c73-a61c-a82933c686bd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.188552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '814419d346854a7fb081b467a3a8fe9a', 'request_id': 'req-d230c612-770c-4c73-a61c-a82933c686bd'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.202285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] The token cache is being invalidate because project 814419d346854a7fb081b467a3a8fe9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:08.202572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.203217 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.204799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1026/2073] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:08 2022] DELETE /identity/v3/projects/814419d346854a7fb081b467a3a8fe9a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:08.209550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562e0a7-a44e-485d-a857-7283ce9a53fa None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:08.221410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562e0a7-a44e-485d-a857-7283ce9a53fa None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:08.310496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbf5e-5b4e-4443-b187-f61618c0cf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:08.311528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbf5e-5b4e-4443-b187-f61618c0cf7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:08.311809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbf5e-5b4e-4443-b187-f61618c0cf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:08.312084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbf5e-5b4e-4443-b187-f61618c0cf7e admin admin] PATH_INFO: `/v3/users/c6c7a82b6f714a18be96f69cf1320ce3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:08.313353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280dbf5e-5b4e-4443-b187-f61618c0cf7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c7a82b6f714a18be96f69cf1320ce3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:08.325325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280dbf5e-5b4e-4443-b187-f61618c0cf7e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:08.411134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-280dbf5e-5b4e-4443-b187-f61618c0cf7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c7a82b6f714a18be96f69cf1320ce3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.411792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-280dbf5e-5b4e-4443-b187-f61618c0cf7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c7a82b6f714a18be96f69cf1320ce3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.426600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-280dbf5e-5b4e-4443-b187-f61618c0cf7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c7a82b6f714a18be96f69cf1320ce3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.427297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-280dbf5e-5b4e-4443-b187-f61618c0cf7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c7a82b6f714a18be96f69cf1320ce3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.437854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-280dbf5e-5b4e-4443-b187-f61618c0cf7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c7a82b6f714a18be96f69cf1320ce3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.450806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1048/2074] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:08 2022] DELETE /identity/v3/users/c6c7a82b6f714a18be96f69cf1320ce3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:08.740405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:08.741895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:08.760899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c612-770c-4c73-a61c-a82933c686bd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:08.826510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:08.827406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:08.827687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:08.827947 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] PATH_INFO: `/v3/projects/c04facab84d54404842657efd8192186` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:08.829138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c04facab84d54404842657efd8192186)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:08.831084 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:08.880262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c04facab84d54404842657efd8192186', 'request_id': 'req-2ed1065a-df7f-4fe8-bea6-4c192af24675'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.886532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c04facab84d54404842657efd8192186', 'request_id': 'req-2ed1065a-df7f-4fe8-bea6-4c192af24675'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.892036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c04facab84d54404842657efd8192186', 'request_id': 'req-2ed1065a-df7f-4fe8-bea6-4c192af24675'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.916507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] The token cache is being invalidate because project c04facab84d54404842657efd8192186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:08.916799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.917463 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2ed1065a-df7f-4fe8-bea6-4c192af24675 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:08.919088 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1027/2075] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:08 2022] DELETE /identity/v3/projects/c04facab84d54404842657efd8192186 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:08.925095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e960924-06d1-43a0-8bff-5cd215934131 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:08.925382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e960924-06d1-43a0-8bff-5cd215934131 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:08.925659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e960924-06d1-43a0-8bff-5cd215934131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:08.945337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1e960924-06d1-43a0-8bff-5cd215934131 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:08.946475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1e960924-06d1-43a0-8bff-5cd215934131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:09.012042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1049/2076] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:09.020147 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2ddae-ec57-4aea-809d-dabd2e333ce4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:09.020429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2ddae-ec57-4aea-809d-dabd2e333ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:09.020709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2ddae-ec57-4aea-809d-dabd2e333ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:09.041940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-51a2ddae-ec57-4aea-809d-dabd2e333ce4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:09.043227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-51a2ddae-ec57-4aea-809d-dabd2e333ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:09.055403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1028/2077] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:09.059860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e960924-06d1-43a0-8bff-5cd215934131 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:09.070240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e960924-06d1-43a0-8bff-5cd215934131 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:09.075467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fa29b-9cd7-4a41-8078-297561127e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:09.076121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fa29b-9cd7-4a41-8078-297561127e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:09.076431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fa29b-9cd7-4a41-8078-297561127e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:09.076705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fa29b-9cd7-4a41-8078-297561127e84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:09.077439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fa29b-9cd7-4a41-8078-297561127e84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:37:09.077758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fa29b-9cd7-4a41-8078-297561127e84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:09.079413 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fa29b-9cd7-4a41-8078-297561127e84 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:09.088406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:09.093797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1050/2078] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:37:09.099757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2ddae-ec57-4aea-809d-dabd2e333ce4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:09.104964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:09.112251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2ddae-ec57-4aea-809d-dabd2e333ce4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:09.118948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394adca-daf2-4b53-97e9-8c3bf0328a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:09.119618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394adca-daf2-4b53-97e9-8c3bf0328a4e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:09.119900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394adca-daf2-4b53-97e9-8c3bf0328a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:09.120196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394adca-daf2-4b53-97e9-8c3bf0328a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:09.120900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394adca-daf2-4b53-97e9-8c3bf0328a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:09.122833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394adca-daf2-4b53-97e9-8c3bf0328a4e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:09.150682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1029/2079] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:37:09 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:09.155046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fa29b-9cd7-4a41-8078-297561127e84 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:09.166122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fa29b-9cd7-4a41-8078-297561127e84 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:09.227151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea94579-4619-40c1-beb4-5df121c6c99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:09.227739 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea94579-4619-40c1-beb4-5df121c6c99e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:09.228018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea94579-4619-40c1-beb4-5df121c6c99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:09.228317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea94579-4619-40c1-beb4-5df121c6c99e admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:09.229113 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea94579-4619-40c1-beb4-5df121c6c99e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:09.231309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea94579-4619-40c1-beb4-5df121c6c99e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:09.254431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1051/2080] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:37:09 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:37:09.258927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394adca-daf2-4b53-97e9-8c3bf0328a4e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:09.269518 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394adca-daf2-4b53-97e9-8c3bf0328a4e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:09.275206 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00f979-58e3-4ddf-a8a2-79e91066091c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:09.275885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00f979-58e3-4ddf-a8a2-79e91066091c admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:09.276175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00f979-58e3-4ddf-a8a2-79e91066091c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:09.276455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00f979-58e3-4ddf-a8a2-79e91066091c admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:09.277203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00f979-58e3-4ddf-a8a2-79e91066091c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:09.278851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00f979-58e3-4ddf-a8a2-79e91066091c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:09.290251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1030/2081] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:09.295052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea94579-4619-40c1-beb4-5df121c6c99e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:09.309530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea94579-4619-40c1-beb4-5df121c6c99e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:09.317118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d135d9d-80ec-4ec6-84cd-a0f520bb64b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:09.318024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d135d9d-80ec-4ec6-84cd-a0f520bb64b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:09.318553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d135d9d-80ec-4ec6-84cd-a0f520bb64b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:09.319075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d135d9d-80ec-4ec6-84cd-a0f520bb64b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:09.319616 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d135d9d-80ec-4ec6-84cd-a0f520bb64b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:09.321228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d135d9d-80ec-4ec6-84cd-a0f520bb64b4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:09.331474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1052/2082] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:09.335793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00f979-58e3-4ddf-a8a2-79e91066091c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:09.347056 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00f979-58e3-4ddf-a8a2-79e91066091c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:09.353041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b1a0e-10f1-4c55-8c37-3cd65a58ff77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:09.353947 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b1a0e-10f1-4c55-8c37-3cd65a58ff77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:09.354268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b1a0e-10f1-4c55-8c37-3cd65a58ff77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:09.354544 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b1a0e-10f1-4c55-8c37-3cd65a58ff77 admin admin] PATH_INFO: `/v3/projects/a4675925d46d42bfa23221229a0b85aa/users/258f273e20a44115ab9b9db0fffde8ce/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:09.370222 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b1a0e-10f1-4c55-8c37-3cd65a58ff77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4675925d46d42bfa23221229a0b85aa, user_id=258f273e20a44115ab9b9db0fffde8ce, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:09.373154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b1a0e-10f1-4c55-8c37-3cd65a58ff77 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:09.391693 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1031/2083] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:09 2022] PUT /identity/v3/projects/a4675925d46d42bfa23221229a0b85aa/users/258f273e20a44115ab9b9db0fffde8ce/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:09.402217 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef7be-4077-4cd5-a3f5-498d423fe28a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:09.402540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef7be-4077-4cd5-a3f5-498d423fe28a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:09.402735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ef7be-4077-4cd5-a3f5-498d423fe28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:09.438598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c82ef7be-4077-4cd5-a3f5-498d423fe28a None None] MFA Rules not processed for user `258f273e20a44115ab9b9db0fffde8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:09.439841 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c82ef7be-4077-4cd5-a3f5-498d423fe28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:09.507865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1053/2084] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:09.519624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b1a0e-10f1-4c55-8c37-3cd65a58ff77 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:09.521681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b97b1a0e-10f1-4c55-8c37-3cd65a58ff77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:09.609449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b1a0e-10f1-4c55-8c37-3cd65a58ff77 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:09.614002 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463986ac-aa89-4c50-a739-423620e8bc71 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd09453b5db645d7b03522f91effb8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:09.615383 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463986ac-aa89-4c50-a739-423620e8bc71 service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:09.615679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463986ac-aa89-4c50-a739-423620e8bc71 service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:09.616046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463986ac-aa89-4c50-a739-423620e8bc71 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:09.625767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463986ac-aa89-4c50-a739-423620e8bc71 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:09.627260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463986ac-aa89-4c50-a739-423620e8bc71 service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:09.637585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1032/2085] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:09 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:18.326269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ef7be-4077-4cd5-a3f5-498d423fe28a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:18.327933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c82ef7be-4077-4cd5-a3f5-498d423fe28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:18.397991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ef7be-4077-4cd5-a3f5-498d423fe28a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:18.402765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:18.403635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:18.403915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:18.404186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] PATH_INFO: `/v3/users/71b5041864d548ffbd6b30922d418445` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:18.405438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] RBAC: Authorizing `identity:delete_user(user_id=71b5041864d548ffbd6b30922d418445)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:18.407427 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:18.418925 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:18.465978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71b5041864d548ffbd6b30922d418445'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:18.466787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71b5041864d548ffbd6b30922d418445'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:18.480186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71b5041864d548ffbd6b30922d418445'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:18.480922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71b5041864d548ffbd6b30922d418445'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:18.489114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71b5041864d548ffbd6b30922d418445'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:18.502677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1054/2086] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:18 2022] DELETE /identity/v3/users/71b5041864d548ffbd6b30922d418445 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:18.507604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0127b51-2ebd-4744-abe8-01d3a7eccb4e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:18.507604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0127b51-2ebd-4744-abe8-01d3a7eccb4e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:18.508068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0127b51-2ebd-4744-abe8-01d3a7eccb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:18.527592 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d0127b51-2ebd-4744-abe8-01d3a7eccb4e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:18.534425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d0127b51-2ebd-4744-abe8-01d3a7eccb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:18.608918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1033/2087] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:18.619832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:18.621539 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:18.716399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691ca7b-1c9b-468d-b6b1-f1f5df93966c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:18.724667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fe5d6-1179-425d-b5ab-e49b004d8fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:18.725354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fe5d6-1179-425d-b5ab-e49b004d8fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:18.725639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fe5d6-1179-425d-b5ab-e49b004d8fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:18.726092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fe5d6-1179-425d-b5ab-e49b004d8fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:18.738171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fe5d6-1179-425d-b5ab-e49b004d8fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:18.740463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fe5d6-1179-425d-b5ab-e49b004d8fb7 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:18.755410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1055/2088] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:19.070724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0127b51-2ebd-4744-abe8-01d3a7eccb4e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:19.072203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d0127b51-2ebd-4744-abe8-01d3a7eccb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:19.085195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0127b51-2ebd-4744-abe8-01d3a7eccb4e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:19.090809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:19.091849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.092137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.095227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] PATH_INFO: `/v3/projects/5ede172a0f1d43ecaa26351640829e7a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.096687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ede172a0f1d43ecaa26351640829e7a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:19.098720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:19.147960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:19.151746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ede172a0f1d43ecaa26351640829e7a', 'request_id': 'req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:19.161280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ede172a0f1d43ecaa26351640829e7a', 'request_id': 'req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:19.167797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ede172a0f1d43ecaa26351640829e7a', 'request_id': 'req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:19.183455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] The token cache is being invalidate because project 5ede172a0f1d43ecaa26351640829e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:19.183744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:19.184763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba84028a-c6b3-415a-b4e0-bf79fafc8d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:19.186439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1034/2089] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:19 2022] DELETE /identity/v3/projects/5ede172a0f1d43ecaa26351640829e7a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:19.191681 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a2d1-1f12-43b4-8033-7197c0ca2f2a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.191972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a2d1-1f12-43b4-8033-7197c0ca2f2a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.192251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa6a2d1-1f12-43b4-8033-7197c0ca2f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.212199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-6aa6a2d1-1f12-43b4-8033-7197c0ca2f2a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:19.213630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6aa6a2d1-1f12-43b4-8033-7197c0ca2f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:19.285051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1056/2090] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:19.293250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28457dcc-57fd-42e2-bf32-c9ed012e9083 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.293810 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28457dcc-57fd-42e2-bf32-c9ed012e9083 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.293922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28457dcc-57fd-42e2-bf32-c9ed012e9083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.318410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-28457dcc-57fd-42e2-bf32-c9ed012e9083 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:19.319685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-28457dcc-57fd-42e2-bf32-c9ed012e9083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:19.330840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1035/2091] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:19.333653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a2d1-1f12-43b4-8033-7197c0ca2f2a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:19.344268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa6a2d1-1f12-43b4-8033-7197c0ca2f2a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:19.349352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a3d2f-21a5-4504-87fe-1e4a6def3dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:19.350062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3d2f-21a5-4504-87fe-1e4a6def3dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.350359 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3d2f-21a5-4504-87fe-1e4a6def3dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.350646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3d2f-21a5-4504-87fe-1e4a6def3dc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.351420 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a3d2f-21a5-4504-87fe-1e4a6def3dc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:37:19.351743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a3d2f-21a5-4504-87fe-1e4a6def3dc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:19.353362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a3d2f-21a5-4504-87fe-1e4a6def3dc9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:19.369198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1057/2092] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:37:19.370730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28457dcc-57fd-42e2-bf32-c9ed012e9083 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:19.380749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28457dcc-57fd-42e2-bf32-c9ed012e9083 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:19.385809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945133a-bbcf-43d1-9909-b4b722db4824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:19.386474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945133a-bbcf-43d1-9909-b4b722db4824 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.386820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945133a-bbcf-43d1-9909-b4b722db4824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.387099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0945133a-bbcf-43d1-9909-b4b722db4824 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.388563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945133a-bbcf-43d1-9909-b4b722db4824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:19.390497 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0945133a-bbcf-43d1-9909-b4b722db4824 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:19.420624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1036/2093] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:37:19 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:19.424884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a3d2f-21a5-4504-87fe-1e4a6def3dc9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:19.435977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a3d2f-21a5-4504-87fe-1e4a6def3dc9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:19.449523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:19.504129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75568268-47d2-4e22-8e48-fa3d1929ed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:19.504820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75568268-47d2-4e22-8e48-fa3d1929ed4a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.505103 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75568268-47d2-4e22-8e48-fa3d1929ed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.505381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75568268-47d2-4e22-8e48-fa3d1929ed4a admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.506148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75568268-47d2-4e22-8e48-fa3d1929ed4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:19.508140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75568268-47d2-4e22-8e48-fa3d1929ed4a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:19.534596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1058/2094] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:37:19 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:19.539053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945133a-bbcf-43d1-9909-b4b722db4824 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:19.551010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0945133a-bbcf-43d1-9909-b4b722db4824 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:19.556298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cf78b-d676-4173-b4cc-82b72cd86fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:19.556979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cf78b-d676-4173-b4cc-82b72cd86fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.557300 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cf78b-d676-4173-b4cc-82b72cd86fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.557536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2cf78b-d676-4173-b4cc-82b72cd86fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.558304 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2cf78b-d676-4173-b4cc-82b72cd86fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:19.560271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2cf78b-d676-4173-b4cc-82b72cd86fa2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:19.574251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1037/2095] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:19.576474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75568268-47d2-4e22-8e48-fa3d1929ed4a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:19.589674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75568268-47d2-4e22-8e48-fa3d1929ed4a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:19.595028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7b7e5-e611-4034-a92d-e1edba1590d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:19.595698 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7b7e5-e611-4034-a92d-e1edba1590d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.595981 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7b7e5-e611-4034-a92d-e1edba1590d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.596258 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7b7e5-e611-4034-a92d-e1edba1590d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.596995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7b7e5-e611-4034-a92d-e1edba1590d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:19.598598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7b7e5-e611-4034-a92d-e1edba1590d1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:19.608933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1059/2096] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:19.613015 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cf78b-d676-4173-b4cc-82b72cd86fa2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:19.623264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2cf78b-d676-4173-b4cc-82b72cd86fa2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:19.627913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf97b1-223b-474d-946d-7cabd6e9ea2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:19.628799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf97b1-223b-474d-946d-7cabd6e9ea2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.629081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf97b1-223b-474d-946d-7cabd6e9ea2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.629354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf97b1-223b-474d-946d-7cabd6e9ea2e admin admin] PATH_INFO: `/v3/projects/27db782381354d39b3b75c4cb0c32e45/users/72394125ebfe47b78b8443d1057448fc/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.645506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf97b1-223b-474d-946d-7cabd6e9ea2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=27db782381354d39b3b75c4cb0c32e45, user_id=72394125ebfe47b78b8443d1057448fc, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:19.648301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf97b1-223b-474d-946d-7cabd6e9ea2e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:19.662882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1038/2097] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:19 2022] PUT /identity/v3/projects/27db782381354d39b3b75c4cb0c32e45/users/72394125ebfe47b78b8443d1057448fc/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:19.673862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204efa95-61a5-4558-b3ca-3e22b1bbea45 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.674084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204efa95-61a5-4558-b3ca-3e22b1bbea45 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.674366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204efa95-61a5-4558-b3ca-3e22b1bbea45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.701141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-204efa95-61a5-4558-b3ca-3e22b1bbea45 None None] MFA Rules not processed for user `72394125ebfe47b78b8443d1057448fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:19.702351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-204efa95-61a5-4558-b3ca-3e22b1bbea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:19.769090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1060/2098] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:19.773932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf97b1-223b-474d-946d-7cabd6e9ea2e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:19.838621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf97b1-223b-474d-946d-7cabd6e9ea2e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:19.844254 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815061cd-a479-4164-9698-58921d1a29e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:19.844919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815061cd-a479-4164-9698-58921d1a29e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.845305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815061cd-a479-4164-9698-58921d1a29e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.845587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815061cd-a479-4164-9698-58921d1a29e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.846281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815061cd-a479-4164-9698-58921d1a29e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:19.848245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815061cd-a479-4164-9698-58921d1a29e1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:19.880214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1039/2099] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:37:19 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:19.884457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204efa95-61a5-4558-b3ca-3e22b1bbea45 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:19.898759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204efa95-61a5-4558-b3ca-3e22b1bbea45 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:19.961919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537a972-4a0e-4c73-a846-ac91174531d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:19.962614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537a972-4a0e-4c73-a846-ac91174531d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:19.962938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537a972-4a0e-4c73-a846-ac91174531d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:19.963218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537a972-4a0e-4c73-a846-ac91174531d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:19.963931 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537a972-4a0e-4c73-a846-ac91174531d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:19.966791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537a972-4a0e-4c73-a846-ac91174531d1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:19.990257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1061/2100] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:37:19 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:19.993896 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815061cd-a479-4164-9698-58921d1a29e1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:20.005612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815061cd-a479-4164-9698-58921d1a29e1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:20.010384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2d4e8-cec0-44d5-afd0-32ced32287e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:20.011094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d4e8-cec0-44d5-afd0-32ced32287e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.011376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d4e8-cec0-44d5-afd0-32ced32287e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.011660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2d4e8-cec0-44d5-afd0-32ced32287e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:20.012448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2d4e8-cec0-44d5-afd0-32ced32287e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:20.014053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2d4e8-cec0-44d5-afd0-32ced32287e0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:20.029822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1040/2101] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:20.031055 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537a972-4a0e-4c73-a846-ac91174531d1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:20.041022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537a972-4a0e-4c73-a846-ac91174531d1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:20.045936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b67d1-8d26-48c7-862a-a7cc6f8b6f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:20.046608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b67d1-8d26-48c7-862a-a7cc6f8b6f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.046932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b67d1-8d26-48c7-862a-a7cc6f8b6f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.047213 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b67d1-8d26-48c7-862a-a7cc6f8b6f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:20.048102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b67d1-8d26-48c7-862a-a7cc6f8b6f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:20.049695 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b67d1-8d26-48c7-862a-a7cc6f8b6f8e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:20.065301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1062/2102] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:37:20.066488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2d4e8-cec0-44d5-afd0-32ced32287e0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:20.076797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2d4e8-cec0-44d5-afd0-32ced32287e0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:20.083449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585de73-b4b0-4a3b-a799-10f3d55e849f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:20.084358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585de73-b4b0-4a3b-a799-10f3d55e849f admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.084639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585de73-b4b0-4a3b-a799-10f3d55e849f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.084912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585de73-b4b0-4a3b-a799-10f3d55e849f admin admin] PATH_INFO: `/v3/projects/8755d8b133ab42299172c5eb6c1d12a2/users/88afd692e6664adab8d98b29f319498c/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:20.100415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585de73-b4b0-4a3b-a799-10f3d55e849f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8755d8b133ab42299172c5eb6c1d12a2, user_id=88afd692e6664adab8d98b29f319498c, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:20.103094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585de73-b4b0-4a3b-a799-10f3d55e849f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:20.131489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1041/2103] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:20 2022] PUT /identity/v3/projects/8755d8b133ab42299172c5eb6c1d12a2/users/88afd692e6664adab8d98b29f319498c/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:20.135078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c18b2-8594-442e-84bd-a2ef5feec308 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.135406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c18b2-8594-442e-84bd-a2ef5feec308 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.135670 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c18b2-8594-442e-84bd-a2ef5feec308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:20.143408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:20.165117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a99c18b2-8594-442e-84bd-a2ef5feec308 None None] MFA Rules not processed for user `88afd692e6664adab8d98b29f319498c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:20.166347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a99c18b2-8594-442e-84bd-a2ef5feec308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:20.235223 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1063/2104] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:20.239916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585de73-b4b0-4a3b-a799-10f3d55e849f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:20.261388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:20.313092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585de73-b4b0-4a3b-a799-10f3d55e849f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:20.319477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b8436-510b-48d0-b58d-adfd0aa509dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:20.320131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8436-510b-48d0-b58d-adfd0aa509dc admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.320410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8436-510b-48d0-b58d-adfd0aa509dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.320688 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8436-510b-48d0-b58d-adfd0aa509dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:20.321407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b8436-510b-48d0-b58d-adfd0aa509dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:20.323380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b8436-510b-48d0-b58d-adfd0aa509dc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:20.354799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1042/2105] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:37:20 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:37:20.359274 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c18b2-8594-442e-84bd-a2ef5feec308 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:20.370895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c18b2-8594-442e-84bd-a2ef5feec308 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:20.439579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d29ac-16e8-4692-9d69-490898af01fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:20.440280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d29ac-16e8-4692-9d69-490898af01fd admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.440564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d29ac-16e8-4692-9d69-490898af01fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.440839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d29ac-16e8-4692-9d69-490898af01fd admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:20.441545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d29ac-16e8-4692-9d69-490898af01fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:20.443563 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d29ac-16e8-4692-9d69-490898af01fd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:20.475291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1064/2106] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:37:20 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:20.480911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b8436-510b-48d0-b58d-adfd0aa509dc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:20.492017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b8436-510b-48d0-b58d-adfd0aa509dc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:20.496819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253bee-c6c7-4aa1-9cbf-51042b3f6e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:20.497870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253bee-c6c7-4aa1-9cbf-51042b3f6e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.498159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253bee-c6c7-4aa1-9cbf-51042b3f6e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.498441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c253bee-c6c7-4aa1-9cbf-51042b3f6e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:20.499217 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c253bee-c6c7-4aa1-9cbf-51042b3f6e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:20.500903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c253bee-c6c7-4aa1-9cbf-51042b3f6e56 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:20.517351 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1043/2107] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:20.521625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d29ac-16e8-4692-9d69-490898af01fd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:20.532474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d29ac-16e8-4692-9d69-490898af01fd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:20.537424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa97f52-3767-43ae-a79c-7701c74ed21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:20.538742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa97f52-3767-43ae-a79c-7701c74ed21d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.538742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa97f52-3767-43ae-a79c-7701c74ed21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.539127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa97f52-3767-43ae-a79c-7701c74ed21d admin admin] PATH_INFO: `/v3/projects/1e4aada2f47a43e29dac094cbbf9c538/users/0123b316bc914b5bb6930281f72fa3ce/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:20.557351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa97f52-3767-43ae-a79c-7701c74ed21d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4aada2f47a43e29dac094cbbf9c538, user_id=0123b316bc914b5bb6930281f72fa3ce, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:20.560202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa97f52-3767-43ae-a79c-7701c74ed21d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:20.575582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1065/2108] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:20 2022] PUT /identity/v3/projects/1e4aada2f47a43e29dac094cbbf9c538/users/0123b316bc914b5bb6930281f72fa3ce/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:20.579873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253bee-c6c7-4aa1-9cbf-51042b3f6e56 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:20.591269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c253bee-c6c7-4aa1-9cbf-51042b3f6e56 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:20.660029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2860f0-3d07-4275-a583-e2547de2eeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:20.660724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2860f0-3d07-4275-a583-e2547de2eeb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.661011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2860f0-3d07-4275-a583-e2547de2eeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.661285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2860f0-3d07-4275-a583-e2547de2eeb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:20.662029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2860f0-3d07-4275-a583-e2547de2eeb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:20.663654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2860f0-3d07-4275-a583-e2547de2eeb1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:20.675015 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1044/2109] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:20.679176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa97f52-3767-43ae-a79c-7701c74ed21d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:20.689968 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa97f52-3767-43ae-a79c-7701c74ed21d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:20.694622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325edc3-792c-4539-a6d2-d53477550612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:20.695568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325edc3-792c-4539-a6d2-d53477550612 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.695917 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325edc3-792c-4539-a6d2-d53477550612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.696783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325edc3-792c-4539-a6d2-d53477550612 admin admin] PATH_INFO: `/v3/projects/1e4aada2f47a43e29dac094cbbf9c538/users/0123b316bc914b5bb6930281f72fa3ce/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:20.699468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325edc3-792c-4539-a6d2-d53477550612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e4aada2f47a43e29dac094cbbf9c538, user_id=0123b316bc914b5bb6930281f72fa3ce, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:20.704102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325edc3-792c-4539-a6d2-d53477550612 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:20.722005 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1066/2110] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:20 2022] PUT /identity/v3/projects/1e4aada2f47a43e29dac094cbbf9c538/users/0123b316bc914b5bb6930281f72fa3ce/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:20.734115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50406a26-4376-4e0f-8af3-ebb5b67ff02e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.734424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50406a26-4376-4e0f-8af3-ebb5b67ff02e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.734727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50406a26-4376-4e0f-8af3-ebb5b67ff02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:20.772058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-50406a26-4376-4e0f-8af3-ebb5b67ff02e None None] MFA Rules not processed for user `0123b316bc914b5bb6930281f72fa3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:20.776442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-50406a26-4376-4e0f-8af3-ebb5b67ff02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:20.866939 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1045/2111] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:20 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:20.879109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325edc3-792c-4539-a6d2-d53477550612 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:20.880938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4325edc3-792c-4539-a6d2-d53477550612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:20.935523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:20.944412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325edc3-792c-4539-a6d2-d53477550612 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:20.948194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c92c90-ccc0-497f-8422-4eacf0f8f9c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:20.948863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c92c90-ccc0-497f-8422-4eacf0f8f9c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:20.949152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c92c90-ccc0-497f-8422-4eacf0f8f9c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:20.949429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c92c90-ccc0-497f-8422-4eacf0f8f9c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:21.019034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c92c90-ccc0-497f-8422-4eacf0f8f9c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:21.020405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c92c90-ccc0-497f-8422-4eacf0f8f9c0 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:21.029989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1067/2112] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:21.315588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3feee-9ad0-4a59-a5a1-af1afc860dfc None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:21.315883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3feee-9ad0-4a59-a5a1-af1afc860dfc None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:21.316184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3feee-9ad0-4a59-a5a1-af1afc860dfc None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:21.319246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1046/2113] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:37:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:37:21.326846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c178be2-2dbe-4e79-83e3-2cc391d90ee0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:21.327146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c178be2-2dbe-4e79-83e3-2cc391d90ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:21.327594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c178be2-2dbe-4e79-83e3-2cc391d90ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:21.355423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9c178be2-2dbe-4e79-83e3-2cc391d90ee0 None None] MFA Rules not processed for user `00a08f31638d4eaab5ca83d6f493db6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:21.356851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9c178be2-2dbe-4e79-83e3-2cc391d90ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:21.444666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1068/2114] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:37:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:21.606770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3feee-9ad0-4a59-a5a1-af1afc860dfc None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:21.608523 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7fe3feee-9ad0-4a59-a5a1-af1afc860dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:21.629667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3feee-9ad0-4a59-a5a1-af1afc860dfc None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:21.635087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffc199-8250-4803-ace7-7fdbc9b7da65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:21.635748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffc199-8250-4803-ace7-7fdbc9b7da65 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:21.636037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffc199-8250-4803-ace7-7fdbc9b7da65 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:21.636293 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffc199-8250-4803-ace7-7fdbc9b7da65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:21.647254 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffc199-8250-4803-ace7-7fdbc9b7da65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:21.648870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffc199-8250-4803-ace7-7fdbc9b7da65 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:21.654438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:21.659880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1047/2115] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:22.170335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c178be2-2dbe-4e79-83e3-2cc391d90ee0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:22.171760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9c178be2-2dbe-4e79-83e3-2cc391d90ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:22.238840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c178be2-2dbe-4e79-83e3-2cc391d90ee0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:22.243876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:22.244760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:22.245042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:22.245313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] PATH_INFO: `/v3/users/6bcb4e12327e4922af739cbb6be41624` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:22.247036 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bcb4e12327e4922af739cbb6be41624)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:22.248753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:22.295641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:22.302486 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bcb4e12327e4922af739cbb6be41624'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.303306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bcb4e12327e4922af739cbb6be41624'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.315180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bcb4e12327e4922af739cbb6be41624'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.315814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bcb4e12327e4922af739cbb6be41624'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.324455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bcb4e12327e4922af739cbb6be41624'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.336826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1069/2116] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:22 2022] DELETE /identity/v3/users/6bcb4e12327e4922af739cbb6be41624 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:22.638422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffc199-8250-4803-ace7-7fdbc9b7da65 service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:22.649177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffc199-8250-4803-ace7-7fdbc9b7da65 service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:22.711501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:22.712378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:22.712661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:22.712939 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] PATH_INFO: `/v3/projects/a39d1fc84c6140a79734b04cd5a70249` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:22.714110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a39d1fc84c6140a79734b04cd5a70249)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:22.716477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:22.766317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a39d1fc84c6140a79734b04cd5a70249', 'request_id': 'req-84fdb4a9-dd6b-412a-b308-3b661ca7019b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.772643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a39d1fc84c6140a79734b04cd5a70249', 'request_id': 'req-84fdb4a9-dd6b-412a-b308-3b661ca7019b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.778917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a39d1fc84c6140a79734b04cd5a70249', 'request_id': 'req-84fdb4a9-dd6b-412a-b308-3b661ca7019b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.795063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] The token cache is being invalidate because project a39d1fc84c6140a79734b04cd5a70249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:22.795356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.799814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.801429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1048/2117] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:22 2022] DELETE /identity/v3/projects/a39d1fc84c6140a79734b04cd5a70249 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:22.805571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:22.807045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:22.820483 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94311a54-a3ba-4573-bb25-8291ba28a37c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:22.892147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc96c3-b877-4c3a-9435-242ffe674662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:22.893392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc96c3-b877-4c3a-9435-242ffe674662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:22.893973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc96c3-b877-4c3a-9435-242ffe674662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:22.894275 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc96c3-b877-4c3a-9435-242ffe674662 admin admin] PATH_INFO: `/v3/users/9fc1fb105ba54d839421ab3f4dccd64c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:22.896330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc96c3-b877-4c3a-9435-242ffe674662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fc1fb105ba54d839421ab3f4dccd64c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:22.898488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc96c3-b877-4c3a-9435-242ffe674662 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:22.958889 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-facc96c3-b877-4c3a-9435-242ffe674662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fc1fb105ba54d839421ab3f4dccd64c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.959622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-facc96c3-b877-4c3a-9435-242ffe674662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fc1fb105ba54d839421ab3f4dccd64c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.974611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-facc96c3-b877-4c3a-9435-242ffe674662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fc1fb105ba54d839421ab3f4dccd64c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.975506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-facc96c3-b877-4c3a-9435-242ffe674662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fc1fb105ba54d839421ab3f4dccd64c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:22.987799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-facc96c3-b877-4c3a-9435-242ffe674662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fc1fb105ba54d839421ab3f4dccd64c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:23.002347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1070/2118] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:22 2022] DELETE /identity/v3/users/9fc1fb105ba54d839421ab3f4dccd64c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:23.264990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:23.266514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:23.281994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdb4a9-dd6b-412a-b308-3b661ca7019b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:23.347563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:23.348454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:23.348759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:23.349031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] PATH_INFO: `/v3/projects/2721b1f85f894eea8171447b77d329b3` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:23.350239 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2721b1f85f894eea8171447b77d329b3)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:23.352200 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:23.389968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:23.405443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2721b1f85f894eea8171447b77d329b3', 'request_id': 'req-3944f7d2-bb72-495f-9c3f-a063da90ce1d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:23.412755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2721b1f85f894eea8171447b77d329b3', 'request_id': 'req-3944f7d2-bb72-495f-9c3f-a063da90ce1d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:23.418739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2721b1f85f894eea8171447b77d329b3', 'request_id': 'req-3944f7d2-bb72-495f-9c3f-a063da90ce1d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:23.438780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] The token cache is being invalidate because project 2721b1f85f894eea8171447b77d329b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:23.439225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:23.440589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3944f7d2-bb72-495f-9c3f-a063da90ce1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:23.441967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1049/2119] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:23 2022] DELETE /identity/v3/projects/2721b1f85f894eea8171447b77d329b3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:23.447108 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657470e-ded9-44c5-bd6e-108d1f85970d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:23.447465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657470e-ded9-44c5-bd6e-108d1f85970d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:23.447806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657470e-ded9-44c5-bd6e-108d1f85970d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:23.468801 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2657470e-ded9-44c5-bd6e-108d1f85970d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:23.470070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2657470e-ded9-44c5-bd6e-108d1f85970d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:23.509601 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:23.561637 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1071/2120] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:23.563375 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33298360-c711-4a21-8007-61d66e26b320 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:23.563674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33298360-c711-4a21-8007-61d66e26b320 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:23.563955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33298360-c711-4a21-8007-61d66e26b320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:23.585414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-33298360-c711-4a21-8007-61d66e26b320 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:23.586673 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-33298360-c711-4a21-8007-61d66e26b320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:23.602359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1050/2121] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:23.606780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657470e-ded9-44c5-bd6e-108d1f85970d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:23.619215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657470e-ded9-44c5-bd6e-108d1f85970d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:23.626166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1cdaa-06f1-4c79-a67d-1f567601c49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:23.627041 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1cdaa-06f1-4c79-a67d-1f567601c49e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:23.627321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1cdaa-06f1-4c79-a67d-1f567601c49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:23.627599 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1cdaa-06f1-4c79-a67d-1f567601c49e admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:23.628338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1cdaa-06f1-4c79-a67d-1f567601c49e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:37:23.628644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1cdaa-06f1-4c79-a67d-1f567601c49e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:23.630293 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba1cdaa-06f1-4c79-a67d-1f567601c49e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:23.649682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1072/2122] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:37:23.651198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33298360-c711-4a21-8007-61d66e26b320 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:23.673979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33298360-c711-4a21-8007-61d66e26b320 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:23.679252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e75e59-0130-40d7-b8c7-f04a61168554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:23.679923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e75e59-0130-40d7-b8c7-f04a61168554 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:23.680203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e75e59-0130-40d7-b8c7-f04a61168554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:23.680479 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e75e59-0130-40d7-b8c7-f04a61168554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:23.681245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e75e59-0130-40d7-b8c7-f04a61168554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:23.683244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e75e59-0130-40d7-b8c7-f04a61168554 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:23.731984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1051/2123] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:37:23 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:23.736216 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1cdaa-06f1-4c79-a67d-1f567601c49e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:23.739866 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e75e59-0130-40d7-b8c7-f04a61168554 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:23.742436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-58e75e59-0130-40d7-b8c7-f04a61168554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:23.748345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1cdaa-06f1-4c79-a67d-1f567601c49e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:23.811805 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e75e59-0130-40d7-b8c7-f04a61168554 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:23.816846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2469fa-34f8-43f6-91b1-fecce6b57ef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:23.817638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2469fa-34f8-43f6-91b1-fecce6b57ef3 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:23.817971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2469fa-34f8-43f6-91b1-fecce6b57ef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:23.818307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2469fa-34f8-43f6-91b1-fecce6b57ef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:23.820384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ca2469fa-34f8-43f6-91b1-fecce6b57ef3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:23.830299 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fa699-e6fa-4e95-84cb-235e3f8d6e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:23.831210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fa699-e6fa-4e95-84cb-235e3f8d6e1e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:23.831502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fa699-e6fa-4e95-84cb-235e3f8d6e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:23.831781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fa699-e6fa-4e95-84cb-235e3f8d6e1e admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:23.832494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fa699-e6fa-4e95-84cb-235e3f8d6e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:23.838077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fa699-e6fa-4e95-84cb-235e3f8d6e1e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:23.863625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1073/2124] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:37:23 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:23.869461 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fa699-e6fa-4e95-84cb-235e3f8d6e1e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:23.877748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fa699-e6fa-4e95-84cb-235e3f8d6e1e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:23.882667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03290ed8-0900-4fc1-b589-c1970123d323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:23.883375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03290ed8-0900-4fc1-b589-c1970123d323 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:23.883665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03290ed8-0900-4fc1-b589-c1970123d323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:23.883946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03290ed8-0900-4fc1-b589-c1970123d323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:23.884689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03290ed8-0900-4fc1-b589-c1970123d323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:23.886305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03290ed8-0900-4fc1-b589-c1970123d323 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:23.900895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1074/2125] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:23.901735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03290ed8-0900-4fc1-b589-c1970123d323 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:23.906447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2469fa-34f8-43f6-91b1-fecce6b57ef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:23.909782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2469fa-34f8-43f6-91b1-fecce6b57ef3 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:23.912505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03290ed8-0900-4fc1-b589-c1970123d323 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:23.918849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdc3f9-39c9-4d10-a76e-af7bc92320d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:23.919516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1052/2126] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:23.920310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdc3f9-39c9-4d10-a76e-af7bc92320d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:23.921130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdc3f9-39c9-4d10-a76e-af7bc92320d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:23.921579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdc3f9-39c9-4d10-a76e-af7bc92320d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:23.922566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdc3f9-39c9-4d10-a76e-af7bc92320d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:23.924495 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdc3f9-39c9-4d10-a76e-af7bc92320d4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:23.941939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1075/2127] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:23.946386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2469fa-34f8-43f6-91b1-fecce6b57ef3 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:23.959163 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2469fa-34f8-43f6-91b1-fecce6b57ef3 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:23.963971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04f550-140a-4e47-b4ee-0b2d66be8566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:23.965442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04f550-140a-4e47-b4ee-0b2d66be8566 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:23.965880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04f550-140a-4e47-b4ee-0b2d66be8566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:23.966274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04f550-140a-4e47-b4ee-0b2d66be8566 admin admin] PATH_INFO: `/v3/projects/b3dfa7ec46ed4d96b70c1d5e70340f8e/users/d794ec64c2ca45d8b7ab25cc0ef10d2b/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:23.987045 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04f550-140a-4e47-b4ee-0b2d66be8566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3dfa7ec46ed4d96b70c1d5e70340f8e, user_id=d794ec64c2ca45d8b7ab25cc0ef10d2b, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:23.989882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04f550-140a-4e47-b4ee-0b2d66be8566 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:24.008977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1053/2128] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:23 2022] PUT /identity/v3/projects/b3dfa7ec46ed4d96b70c1d5e70340f8e/users/d794ec64c2ca45d8b7ab25cc0ef10d2b/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:24.020645 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abf08e-1611-4d9f-b113-b1a32dd69c53 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:24.020932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abf08e-1611-4d9f-b113-b1a32dd69c53 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:24.021206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abf08e-1611-4d9f-b113-b1a32dd69c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:24.062882 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-28abf08e-1611-4d9f-b113-b1a32dd69c53 None None] MFA Rules not processed for user `d794ec64c2ca45d8b7ab25cc0ef10d2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:24.064600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-28abf08e-1611-4d9f-b113-b1a32dd69c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:24.149474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1076/2129] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:24.160727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04f550-140a-4e47-b4ee-0b2d66be8566 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:24.162425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bd04f550-140a-4e47-b4ee-0b2d66be8566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:24.190757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:24.228696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04f550-140a-4e47-b4ee-0b2d66be8566 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:24.233318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef36582-4a7b-4566-9cc3-6370334a213e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:24.234179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef36582-4a7b-4566-9cc3-6370334a213e service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:24.234557 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef36582-4a7b-4566-9cc3-6370334a213e service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:24.235519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef36582-4a7b-4566-9cc3-6370334a213e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:24.245405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef36582-4a7b-4566-9cc3-6370334a213e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:24.247291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef36582-4a7b-4566-9cc3-6370334a213e service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:24.259229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1054/2130] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:24.599128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abf08e-1611-4d9f-b113-b1a32dd69c53 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:24.611745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abf08e-1611-4d9f-b113-b1a32dd69c53 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:24.614006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbd427-6524-44c2-8bc6-cdc6fe3d5964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:24.614667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbd427-6524-44c2-8bc6-cdc6fe3d5964 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:24.614987 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbd427-6524-44c2-8bc6-cdc6fe3d5964 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:24.615325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbd427-6524-44c2-8bc6-cdc6fe3d5964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:24.617239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-15fbd427-6524-44c2-8bc6-cdc6fe3d5964 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:24.639447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:24.692441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbd427-6524-44c2-8bc6-cdc6fe3d5964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:24.693894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fbd427-6524-44c2-8bc6-cdc6fe3d5964 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:24.703904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1077/2131] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:34.259321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=16 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:34.700260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=21 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:38.114231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef36582-4a7b-4566-9cc3-6370334a213e service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:38.115802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2ef36582-4a7b-4566-9cc3-6370334a213e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:38.188091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef36582-4a7b-4566-9cc3-6370334a213e service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:38.193194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:38.194249 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:38.194534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:38.194843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] PATH_INFO: `/v3/users/72394125ebfe47b78b8443d1057448fc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:38.196156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72394125ebfe47b78b8443d1057448fc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:38.198251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:38.269260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72394125ebfe47b78b8443d1057448fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:38.270472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72394125ebfe47b78b8443d1057448fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:38.288243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72394125ebfe47b78b8443d1057448fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:38.289287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72394125ebfe47b78b8443d1057448fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:38.310937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72394125ebfe47b78b8443d1057448fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:38.326737 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1055/2132] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:38 2022] DELETE /identity/v3/users/72394125ebfe47b78b8443d1057448fc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:38.332048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375cd7f-68b9-4758-9aa3-38023c1dbee1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:38.332682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375cd7f-68b9-4758-9aa3-38023c1dbee1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:38.332682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375cd7f-68b9-4758-9aa3-38023c1dbee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:38.356760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b375cd7f-68b9-4758-9aa3-38023c1dbee1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:38.358231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b375cd7f-68b9-4758-9aa3-38023c1dbee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:38.485669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1078/2133] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:38.493154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:38.494631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:38.569480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c62efe-80f3-47df-8416-e3d381ce37a0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:38.576346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17d861-b92e-433a-8f4a-524aa3fedff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:38.577161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17d861-b92e-433a-8f4a-524aa3fedff3 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:38.577515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17d861-b92e-433a-8f4a-524aa3fedff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:38.577877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17d861-b92e-433a-8f4a-524aa3fedff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:38.603253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc17d861-b92e-433a-8f4a-524aa3fedff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:38.605436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc17d861-b92e-433a-8f4a-524aa3fedff3 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:38.619461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1056/2134] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:38.919342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375cd7f-68b9-4758-9aa3-38023c1dbee1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:38.921381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b375cd7f-68b9-4758-9aa3-38023c1dbee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:38.943043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375cd7f-68b9-4758-9aa3-38023c1dbee1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:38.948429 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:38.949329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:38.949610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:38.949883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] PATH_INFO: `/v3/projects/27db782381354d39b3b75c4cb0c32e45` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:38.951225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=27db782381354d39b3b75c4cb0c32e45)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:38.953162 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:38.999788 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17d861-b92e-433a-8f4a-524aa3fedff3 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:39.001234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bc17d861-b92e-433a-8f4a-524aa3fedff3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:39.003929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27db782381354d39b3b75c4cb0c32e45', 'request_id': 'req-b6ce3a11-72f5-47c5-aed8-725538a277fc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.010280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27db782381354d39b3b75c4cb0c32e45', 'request_id': 'req-b6ce3a11-72f5-47c5-aed8-725538a277fc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.014426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17d861-b92e-433a-8f4a-524aa3fedff3 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:39.016751 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27db782381354d39b3b75c4cb0c32e45', 'request_id': 'req-b6ce3a11-72f5-47c5-aed8-725538a277fc'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.032882 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] The token cache is being invalidate because project 27db782381354d39b3b75c4cb0c32e45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:39.033295 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.034116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.034950 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:39.036533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1079/2135] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:38 2022] DELETE /identity/v3/projects/27db782381354d39b3b75c4cb0c32e45 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:39.041492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:39.043077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:39.084491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dea8b4-cbd5-41af-8819-5af0b0c56341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:39.085410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dea8b4-cbd5-41af-8819-5af0b0c56341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:39.085693 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dea8b4-cbd5-41af-8819-5af0b0c56341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:39.085969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dea8b4-cbd5-41af-8819-5af0b0c56341 admin admin] PATH_INFO: `/v3/users/d794ec64c2ca45d8b7ab25cc0ef10d2b` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:39.087302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dea8b4-cbd5-41af-8819-5af0b0c56341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d794ec64c2ca45d8b7ab25cc0ef10d2b)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:39.089257 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dea8b4-cbd5-41af-8819-5af0b0c56341 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:39.118107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3a11-72f5-47c5-aed8-725538a277fc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:39.123475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:39.124628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:39.124912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:39.125242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] PATH_INFO: `/v3/users/88afd692e6664adab8d98b29f319498c` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:39.126854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88afd692e6664adab8d98b29f319498c)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:39.128937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:39.139384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-64dea8b4-cbd5-41af-8819-5af0b0c56341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd794ec64c2ca45d8b7ab25cc0ef10d2b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.140043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-64dea8b4-cbd5-41af-8819-5af0b0c56341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd794ec64c2ca45d8b7ab25cc0ef10d2b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.152109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-64dea8b4-cbd5-41af-8819-5af0b0c56341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd794ec64c2ca45d8b7ab25cc0ef10d2b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.152830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-64dea8b4-cbd5-41af-8819-5af0b0c56341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd794ec64c2ca45d8b7ab25cc0ef10d2b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.159646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:39.160332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-64dea8b4-cbd5-41af-8819-5af0b0c56341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd794ec64c2ca45d8b7ab25cc0ef10d2b'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.177031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1057/2136] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:38 2022] DELETE /identity/v3/users/d794ec64c2ca45d8b7ab25cc0ef10d2b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:39.179032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41258b25-ab22-45bf-98e3-eaee13933e33 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:39.179430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41258b25-ab22-45bf-98e3-eaee13933e33 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:39.179710 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41258b25-ab22-45bf-98e3-eaee13933e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:39.183958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88afd692e6664adab8d98b29f319498c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.184785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88afd692e6664adab8d98b29f319498c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.199545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88afd692e6664adab8d98b29f319498c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.199703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-41258b25-ab22-45bf-98e3-eaee13933e33 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:39.200311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88afd692e6664adab8d98b29f319498c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.200843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-41258b25-ab22-45bf-98e3-eaee13933e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:39.216034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88afd692e6664adab8d98b29f319498c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.237087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1080/2137] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:39 2022] DELETE /identity/v3/users/88afd692e6664adab8d98b29f319498c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:39.282756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1058/2138] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:39.302299 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:39.303812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:39.366095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646c15d-953b-422e-b8ed-dd0c03cb7b61 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:39.370042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b994a3-287d-4edc-8bac-702398646b9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:39.370723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b994a3-287d-4edc-8bac-702398646b9e service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:39.371012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b994a3-287d-4edc-8bac-702398646b9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:39.371295 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b994a3-287d-4edc-8bac-702398646b9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:39.436172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b994a3-287d-4edc-8bac-702398646b9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:39.437620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b994a3-287d-4edc-8bac-702398646b9e service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:39.448475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1081/2139] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:39.572212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41258b25-ab22-45bf-98e3-eaee13933e33 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:39.573932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-41258b25-ab22-45bf-98e3-eaee13933e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:39.587389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41258b25-ab22-45bf-98e3-eaee13933e33 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:39.593237 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:39.594055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:39.594335 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:39.594613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] PATH_INFO: `/v3/projects/8755d8b133ab42299172c5eb6c1d12a2` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:39.595880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8755d8b133ab42299172c5eb6c1d12a2)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:39.598049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:39.644702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:39.659753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8755d8b133ab42299172c5eb6c1d12a2', 'request_id': 'req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.670408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8755d8b133ab42299172c5eb6c1d12a2', 'request_id': 'req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.678517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8755d8b133ab42299172c5eb6c1d12a2', 'request_id': 'req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.695678 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] The token cache is being invalidate because project 8755d8b133ab42299172c5eb6c1d12a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:39.696207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.697313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.700386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1059/2140] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:39 2022] DELETE /identity/v3/projects/8755d8b133ab42299172c5eb6c1d12a2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:39.704395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b994a3-287d-4edc-8bac-702398646b9e service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:39.714581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b994a3-287d-4edc-8bac-702398646b9e service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:39.762639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:39.773400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed78bb-4cab-44ac-acb4-84baaac1aea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:39.774456 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed78bb-4cab-44ac-acb4-84baaac1aea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:39.774676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed78bb-4cab-44ac-acb4-84baaac1aea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:39.775106 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed78bb-4cab-44ac-acb4-84baaac1aea5 admin admin] PATH_INFO: `/v3/users/0123b316bc914b5bb6930281f72fa3ce` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:39.776674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed78bb-4cab-44ac-acb4-84baaac1aea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0123b316bc914b5bb6930281f72fa3ce)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:39.779227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed78bb-4cab-44ac-acb4-84baaac1aea5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:39.798229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:39.800788 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:39.824410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833bb31-9ff3-4c7e-b3d6-97f466ce14b4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:39.831920 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:39.832995 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:39.833143 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:39.833388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] PATH_INFO: `/v3/projects/b3dfa7ec46ed4d96b70c1d5e70340f8e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:39.834637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3dfa7ec46ed4d96b70c1d5e70340f8e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:39.836700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:39.846567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-44ed78bb-4cab-44ac-acb4-84baaac1aea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0123b316bc914b5bb6930281f72fa3ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.847186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-44ed78bb-4cab-44ac-acb4-84baaac1aea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0123b316bc914b5bb6930281f72fa3ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.859034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-44ed78bb-4cab-44ac-acb4-84baaac1aea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0123b316bc914b5bb6930281f72fa3ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.859700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-44ed78bb-4cab-44ac-acb4-84baaac1aea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0123b316bc914b5bb6930281f72fa3ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.868786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-44ed78bb-4cab-44ac-acb4-84baaac1aea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0123b316bc914b5bb6930281f72fa3ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.881795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1082/2141] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:39 2022] DELETE /identity/v3/users/0123b316bc914b5bb6930281f72fa3ce => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:39.888536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3dfa7ec46ed4d96b70c1d5e70340f8e', 'request_id': 'req-b8ee615b-c152-4214-8084-414cb17ac3a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.895061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3dfa7ec46ed4d96b70c1d5e70340f8e', 'request_id': 'req-b8ee615b-c152-4214-8084-414cb17ac3a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.901187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3dfa7ec46ed4d96b70c1d5e70340f8e', 'request_id': 'req-b8ee615b-c152-4214-8084-414cb17ac3a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.916117 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] The token cache is being invalidate because project b3dfa7ec46ed4d96b70c1d5e70340f8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:39.916412 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.917094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b8ee615b-c152-4214-8084-414cb17ac3a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:39.919516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1060/2142] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:39 2022] DELETE /identity/v3/projects/b3dfa7ec46ed4d96b70c1d5e70340f8e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:39.924354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d52f5-abce-4a59-ab15-1d4510fac73b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:39.924953 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d52f5-abce-4a59-ab15-1d4510fac73b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:39.925248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d52f5-abce-4a59-ab15-1d4510fac73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:39.954134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-209d52f5-abce-4a59-ab15-1d4510fac73b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:39.955444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-209d52f5-abce-4a59-ab15-1d4510fac73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:40.037812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1083/2143] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:40.047980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66511eb-bd21-4d6b-b3de-5495623eb9b4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.048292 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66511eb-bd21-4d6b-b3de-5495623eb9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.048551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66511eb-bd21-4d6b-b3de-5495623eb9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.066733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f66511eb-bd21-4d6b-b3de-5495623eb9b4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:40.067953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f66511eb-bd21-4d6b-b3de-5495623eb9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:40.076070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1061/2144] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:40.080402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d52f5-abce-4a59-ab15-1d4510fac73b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:40.091566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d52f5-abce-4a59-ab15-1d4510fac73b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:40.098650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f97d4-b423-4a63-99f4-14ed2d35806c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:40.099389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f97d4-b423-4a63-99f4-14ed2d35806c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.099733 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f97d4-b423-4a63-99f4-14ed2d35806c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.100065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0f97d4-b423-4a63-99f4-14ed2d35806c admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.101007 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f97d4-b423-4a63-99f4-14ed2d35806c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:37:40.101380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f97d4-b423-4a63-99f4-14ed2d35806c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:40.103095 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0f97d4-b423-4a63-99f4-14ed2d35806c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:40.120271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1084/2145] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:37:40.121970 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66511eb-bd21-4d6b-b3de-5495623eb9b4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:40.143575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66511eb-bd21-4d6b-b3de-5495623eb9b4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:40.158535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78571bb7-b8a6-4cfc-9fc0-8e6c845cf795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:40.159358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78571bb7-b8a6-4cfc-9fc0-8e6c845cf795 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.159642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78571bb7-b8a6-4cfc-9fc0-8e6c845cf795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.159960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78571bb7-b8a6-4cfc-9fc0-8e6c845cf795 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.160614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78571bb7-b8a6-4cfc-9fc0-8e6c845cf795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:40.163444 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78571bb7-b8a6-4cfc-9fc0-8e6c845cf795 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:40.206229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1062/2146] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:37:40 2022] POST /identity/v3/projects => generated 399 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:40.208147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f97d4-b423-4a63-99f4-14ed2d35806c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:40.223770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0f97d4-b423-4a63-99f4-14ed2d35806c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:40.300742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d27f28-4530-4ff5-8123-d8179e52e2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:40.302871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d27f28-4530-4ff5-8123-d8179e52e2ca admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.303196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d27f28-4530-4ff5-8123-d8179e52e2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.303485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d27f28-4530-4ff5-8123-d8179e52e2ca admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.304358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d27f28-4530-4ff5-8123-d8179e52e2ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:40.306313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d27f28-4530-4ff5-8123-d8179e52e2ca admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:40.313979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:40.339214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1085/2147] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:37:40 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:37:40.344475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78571bb7-b8a6-4cfc-9fc0-8e6c845cf795 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:40.347768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d27f28-4530-4ff5-8123-d8179e52e2ca admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:40.349837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b1d27f28-4530-4ff5-8123-d8179e52e2ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:40.359205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78571bb7-b8a6-4cfc-9fc0-8e6c845cf795 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:40.364358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d27f28-4530-4ff5-8123-d8179e52e2ca admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:40.374752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:40.375392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37bbd6-598e-4376-b962-61db879298dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:40.375982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37bbd6-598e-4376-b962-61db879298dd admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.376377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37bbd6-598e-4376-b962-61db879298dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.376539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37bbd6-598e-4376-b962-61db879298dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.377340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37bbd6-598e-4376-b962-61db879298dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:40.378064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.378381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.378930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] PATH_INFO: `/v3/projects/1e4aada2f47a43e29dac094cbbf9c538` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.382378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37bbd6-598e-4376-b962-61db879298dd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:40.382777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e4aada2f47a43e29dac094cbbf9c538)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:40.384714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:40.396580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1063/2148] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:40.400378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37bbd6-598e-4376-b962-61db879298dd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:40.410858 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37bbd6-598e-4376-b962-61db879298dd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:40.415906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf51f7f-434a-4272-a287-9b1e11b96c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:40.416633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf51f7f-434a-4272-a287-9b1e11b96c52 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.416915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf51f7f-434a-4272-a287-9b1e11b96c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.417187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf51f7f-434a-4272-a287-9b1e11b96c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.417948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf51f7f-434a-4272-a287-9b1e11b96c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:40.419610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf51f7f-434a-4272-a287-9b1e11b96c52 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:40.440390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1064/2149] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:37:40.444475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf51f7f-434a-4272-a287-9b1e11b96c52 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:40.449866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e4aada2f47a43e29dac094cbbf9c538', 'request_id': 'req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:40.465188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf51f7f-434a-4272-a287-9b1e11b96c52 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:40.465580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e4aada2f47a43e29dac094cbbf9c538', 'request_id': 'req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:40.473566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e4aada2f47a43e29dac094cbbf9c538', 'request_id': 'req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:40.494786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] The token cache is being invalidate because project 1e4aada2f47a43e29dac094cbbf9c538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:40.495127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:40.495767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1abd9e34-a055-443a-9a22-bd9c0f5fdf3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:40.497533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1086/2150] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:40 2022] DELETE /identity/v3/projects/1e4aada2f47a43e29dac094cbbf9c538 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:40.508710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f229-fa89-4db0-b2a1-8093dff48f9f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.509350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f229-fa89-4db0-b2a1-8093dff48f9f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.509728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f229-fa89-4db0-b2a1-8093dff48f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.532504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-b108f229-fa89-4db0-b2a1-8093dff48f9f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:40.533850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b108f229-fa89-4db0-b2a1-8093dff48f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:40.537804 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:40.546608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5c346-a0cd-4198-b2c4-4656302d2ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:40.547644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5c346-a0cd-4198-b2c4-4656302d2ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.548258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5c346-a0cd-4198-b2c4-4656302d2ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.549369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5c346-a0cd-4198-b2c4-4656302d2ef3 admin admin] PATH_INFO: `/v3/projects/467375e5e3e447f1b73f5b26bd81ec71/users/86de1d29eeed434f9aebdefe57e25c47/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.567704 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5c346-a0cd-4198-b2c4-4656302d2ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=467375e5e3e447f1b73f5b26bd81ec71, user_id=86de1d29eeed434f9aebdefe57e25c47, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:40.570842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5c346-a0cd-4198-b2c4-4656302d2ef3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:40.591850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1065/2151] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:40 2022] PUT /identity/v3/projects/467375e5e3e447f1b73f5b26bd81ec71/users/86de1d29eeed434f9aebdefe57e25c47/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:40.603246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97330-11f7-43d8-9456-c62aa95595b3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.603441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97330-11f7-43d8-9456-c62aa95595b3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.603816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f97330-11f7-43d8-9456-c62aa95595b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.619457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1087/2152] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:37:40.628113 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148653e1-aafd-44f6-8a4a-139d12c005e6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.628398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148653e1-aafd-44f6-8a4a-139d12c005e6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.628676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148653e1-aafd-44f6-8a4a-139d12c005e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.632477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-34f97330-11f7-43d8-9456-c62aa95595b3 None None] MFA Rules not processed for user `86de1d29eeed434f9aebdefe57e25c47`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:40.633699 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-34f97330-11f7-43d8-9456-c62aa95595b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:40.648465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-148653e1-aafd-44f6-8a4a-139d12c005e6 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:40.650049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-148653e1-aafd-44f6-8a4a-139d12c005e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:40.711868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1066/2153] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:37:40.728902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1088/2154] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:37:40.733233 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f97330-11f7-43d8-9456-c62aa95595b3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:40.745743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f97330-11f7-43d8-9456-c62aa95595b3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:40.750849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd4874-b510-408b-8e07-e0415ec9c9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:40.751516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd4874-b510-408b-8e07-e0415ec9c9ef admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.751799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd4874-b510-408b-8e07-e0415ec9c9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.752076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd4874-b510-408b-8e07-e0415ec9c9ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.753175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd4874-b510-408b-8e07-e0415ec9c9ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:37:40.753418 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd4874-b510-408b-8e07-e0415ec9c9ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:40.755058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bd4874-b510-408b-8e07-e0415ec9c9ef admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:40.767355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1067/2155] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:37:40.772296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148653e1-aafd-44f6-8a4a-139d12c005e6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:40.786094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148653e1-aafd-44f6-8a4a-139d12c005e6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:40.799812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eff3b9-6b4f-40a7-9580-a45f53abb828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:40.801125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff3b9-6b4f-40a7-9580-a45f53abb828 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.801612 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff3b9-6b4f-40a7-9580-a45f53abb828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.802033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff3b9-6b4f-40a7-9580-a45f53abb828 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.802953 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eff3b9-6b4f-40a7-9580-a45f53abb828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:40.805598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eff3b9-6b4f-40a7-9580-a45f53abb828 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:40.818155 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:40.842036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd4874-b510-408b-8e07-e0415ec9c9ef admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:40.844393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-70bd4874-b510-408b-8e07-e0415ec9c9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:40.855207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1089/2156] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:37:40 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:40.857182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eff3b9-6b4f-40a7-9580-a45f53abb828 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:40.931691 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bd4874-b510-408b-8e07-e0415ec9c9ef admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:40.955070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eff3b9-6b4f-40a7-9580-a45f53abb828 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:40.960262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e2f8d-9266-4f38-b668-26ab29c5b3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:40.960561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e2f8d-9266-4f38-b668-26ab29c5b3de admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.960831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e2f8d-9266-4f38-b668-26ab29c5b3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.961164 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91e2f8d-9266-4f38-b668-26ab29c5b3de admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.961871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e2f8d-9266-4f38-b668-26ab29c5b3de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:40.963845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91e2f8d-9266-4f38-b668-26ab29c5b3de admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:40.990803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1090/2157] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:37:40 2022] POST /identity/v3/users => generated 286 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:40.991007 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585edf5-0fc8-4963-a2ee-cb15f1dc181d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:40.991007 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585edf5-0fc8-4963-a2ee-cb15f1dc181d service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:40.991007 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585edf5-0fc8-4963-a2ee-cb15f1dc181d service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:40.991007 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585edf5-0fc8-4963-a2ee-cb15f1dc181d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:40.992588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e2f8d-9266-4f38-b668-26ab29c5b3de admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:41.003066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91e2f8d-9266-4f38-b668-26ab29c5b3de admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:41.008773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e52676-4971-4c9d-970c-c6df4b0d9824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:41.009881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e52676-4971-4c9d-970c-c6df4b0d9824 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:41.010319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e52676-4971-4c9d-970c-c6df4b0d9824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:41.010771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e52676-4971-4c9d-970c-c6df4b0d9824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:41.011655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e52676-4971-4c9d-970c-c6df4b0d9824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:41.013280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e52676-4971-4c9d-970c-c6df4b0d9824 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:41.020671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:41.023963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1091/2158] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:41.028434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e52676-4971-4c9d-970c-c6df4b0d9824 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:41.037977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e52676-4971-4c9d-970c-c6df4b0d9824 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:41.042585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a77660-e410-469c-9825-2d9f1c5cf380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:41.043364 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a77660-e410-469c-9825-2d9f1c5cf380 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:41.043555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a77660-e410-469c-9825-2d9f1c5cf380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:41.043832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a77660-e410-469c-9825-2d9f1c5cf380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:41.044561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a77660-e410-469c-9825-2d9f1c5cf380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:41.046152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a77660-e410-469c-9825-2d9f1c5cf380 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:41.054215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585edf5-0fc8-4963-a2ee-cb15f1dc181d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:41.055688 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585edf5-0fc8-4963-a2ee-cb15f1dc181d service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:41.058720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1092/2159] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:37:41.060938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a77660-e410-469c-9825-2d9f1c5cf380 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:41.065126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1068/2160] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:37:41.070830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a77660-e410-469c-9825-2d9f1c5cf380 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:41.075781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa27325-49bd-4e6b-b8ca-56d1be3e6f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:41.076676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27325-49bd-4e6b-b8ca-56d1be3e6f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:41.076954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27325-49bd-4e6b-b8ca-56d1be3e6f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:41.077226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27325-49bd-4e6b-b8ca-56d1be3e6f87 admin admin] PATH_INFO: `/v3/projects/54c1ecc58b824fb7a32b9f093de9a5e7/users/adcd31f61c374c58bc56a6e74f13a9d4/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:41.092638 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27325-49bd-4e6b-b8ca-56d1be3e6f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c1ecc58b824fb7a32b9f093de9a5e7, user_id=adcd31f61c374c58bc56a6e74f13a9d4, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:41.095338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27325-49bd-4e6b-b8ca-56d1be3e6f87 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:41.123822 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1093/2161] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:41 2022] PUT /identity/v3/projects/54c1ecc58b824fb7a32b9f093de9a5e7/users/adcd31f61c374c58bc56a6e74f13a9d4/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:41.133978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6311b-f77b-45c9-b913-026bfeb426df None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:41.134269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6311b-f77b-45c9-b913-026bfeb426df None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:41.134548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c6311b-f77b-45c9-b913-026bfeb426df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:41.174965 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-90c6311b-f77b-45c9-b913-026bfeb426df None None] MFA Rules not processed for user `adcd31f61c374c58bc56a6e74f13a9d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:41.176189 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-90c6311b-f77b-45c9-b913-026bfeb426df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:41.287087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1069/2162] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:41.296381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa27325-49bd-4e6b-b8ca-56d1be3e6f87 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:41.298184 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-eaa27325-49bd-4e6b-b8ca-56d1be3e6f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:41.383795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa27325-49bd-4e6b-b8ca-56d1be3e6f87 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:41.387631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43973d9-ca26-4d62-924a-b7c357e82a50 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd09453b5db645d7b03522f91effb8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:41.388292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43973d9-ca26-4d62-924a-b7c357e82a50 service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:41.388576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43973d9-ca26-4d62-924a-b7c357e82a50 service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:41.388998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43973d9-ca26-4d62-924a-b7c357e82a50 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:41.398990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43973d9-ca26-4d62-924a-b7c357e82a50 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:41.400656 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43973d9-ca26-4d62-924a-b7c357e82a50 service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:41.411075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1094/2163] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:43.711724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5962c89-bd19-443b-9336-ffba1ceb8686 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:43.712228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5962c89-bd19-443b-9336-ffba1ceb8686 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:43.712330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5962c89-bd19-443b-9336-ffba1ceb8686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:43.739155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f5962c89-bd19-443b-9336-ffba1ceb8686 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:43.740667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f5962c89-bd19-443b-9336-ffba1ceb8686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:43.797683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1070/2164] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:37:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:43.844688 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43973d9-ca26-4d62-924a-b7c357e82a50 service glance] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:43.846244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b43973d9-ca26-4d62-924a-b7c357e82a50 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:43.856777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:43.929158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43973d9-ca26-4d62-924a-b7c357e82a50 service glance] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:43.934445 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8769082-bb49-4ac2-8024-0d811cb7ecbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:43.935142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8769082-bb49-4ac2-8024-0d811cb7ecbf service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:43.935424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8769082-bb49-4ac2-8024-0d811cb7ecbf service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:43.935697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8769082-bb49-4ac2-8024-0d811cb7ecbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:43.944541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8769082-bb49-4ac2-8024-0d811cb7ecbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:43.946284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8769082-bb49-4ac2-8024-0d811cb7ecbf service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:43.955231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1095/2165] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:37:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:37:44.528923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fc082-214e-4c09-94c0-9402f45c0f1c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:44.529216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fc082-214e-4c09-94c0-9402f45c0f1c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:44.529501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fc082-214e-4c09-94c0-9402f45c0f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:44.548647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8c8fc082-214e-4c09-94c0-9402f45c0f1c None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:44.549875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8c8fc082-214e-4c09-94c0-9402f45c0f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:44.557269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1071/2166] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jun 29 08:37:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:44.569862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8769082-bb49-4ac2-8024-0d811cb7ecbf service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:44.580291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8769082-bb49-4ac2-8024-0d811cb7ecbf service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:44.585469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd68ec-6ec3-4dfa-9aee-bec057b56ab5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:44.586180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd68ec-6ec3-4dfa-9aee-bec057b56ab5 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:44.586541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd68ec-6ec3-4dfa-9aee-bec057b56ab5 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:44.586839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd68ec-6ec3-4dfa-9aee-bec057b56ab5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:44.596863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd68ec-6ec3-4dfa-9aee-bec057b56ab5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:44.598449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd68ec-6ec3-4dfa-9aee-bec057b56ab5 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:44.607050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1096/2167] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:37:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:37:44.827234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53432fa-4f7c-421b-a2e1-4adc50661baa None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:44.828049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53432fa-4f7c-421b-a2e1-4adc50661baa None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:44.828492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53432fa-4f7c-421b-a2e1-4adc50661baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:44.848305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a53432fa-4f7c-421b-a2e1-4adc50661baa None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:44.849547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a53432fa-4f7c-421b-a2e1-4adc50661baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:44.857263 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1072/2168] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:37:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:44.871433 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd68ec-6ec3-4dfa-9aee-bec057b56ab5 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:44.883277 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd68ec-6ec3-4dfa-9aee-bec057b56ab5 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:44.889375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c9261-cca1-4a34-a543-4a6e569b58b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:44.890060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c9261-cca1-4a34-a543-4a6e569b58b0 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:44.890344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c9261-cca1-4a34-a543-4a6e569b58b0 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:44.890647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c9261-cca1-4a34-a543-4a6e569b58b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:44.900064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c9261-cca1-4a34-a543-4a6e569b58b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:44.902147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c9261-cca1-4a34-a543-4a6e569b58b0 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:44.912707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1097/2169] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:37:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:37:45.311634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0bdff-6335-4127-85b8-64b02146de70 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:45.311930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0bdff-6335-4127-85b8-64b02146de70 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:45.312211 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0bdff-6335-4127-85b8-64b02146de70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:45.329978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e6d0bdff-6335-4127-85b8-64b02146de70 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:45.331319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e6d0bdff-6335-4127-85b8-64b02146de70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:45.339012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1073/2170] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jun 29 08:37:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:45.352305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c9261-cca1-4a34-a543-4a6e569b58b0 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:45.363665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c9261-cca1-4a34-a543-4a6e569b58b0 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:45.368748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585990c8-fda4-4d19-a66a-2486b3cff991 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:45.369435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585990c8-fda4-4d19-a66a-2486b3cff991 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:45.369731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585990c8-fda4-4d19-a66a-2486b3cff991 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:45.370021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585990c8-fda4-4d19-a66a-2486b3cff991 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:45.390269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585990c8-fda4-4d19-a66a-2486b3cff991 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:45.390269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585990c8-fda4-4d19-a66a-2486b3cff991 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:45.398284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1098/2171] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:37:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:37:48.877151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7decbf6-6037-4af8-aeef-1592a3406a97 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:48.877754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7decbf6-6037-4af8-aeef-1592a3406a97 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:48.878065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7decbf6-6037-4af8-aeef-1592a3406a97 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:48.879881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1074/2172] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:37:48.889127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873947ce-0742-453b-88de-70331a69fd49 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:48.889663 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873947ce-0742-453b-88de-70331a69fd49 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:48.889663 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873947ce-0742-453b-88de-70331a69fd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:48.913084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-873947ce-0742-453b-88de-70331a69fd49 None None] MFA Rules not processed for user `00a08f31638d4eaab5ca83d6f493db6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:48.914633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-873947ce-0742-453b-88de-70331a69fd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:48.955166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7decbf6-6037-4af8-aeef-1592a3406a97 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:48.957132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b7decbf6-6037-4af8-aeef-1592a3406a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:48.981420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:48.985030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1099/2173] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:37:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:48.996349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873947ce-0742-453b-88de-70331a69fd49 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:48.997939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-873947ce-0742-453b-88de-70331a69fd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:49.011302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873947ce-0742-453b-88de-70331a69fd49 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:49.016524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed7d9f-226c-4d83-ad25-3657d3d6b5b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:49.017175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed7d9f-226c-4d83-ad25-3657d3d6b5b8 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:49.017463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed7d9f-226c-4d83-ad25-3657d3d6b5b8 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:49.017735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed7d9f-226c-4d83-ad25-3657d3d6b5b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:49.032057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed7d9f-226c-4d83-ad25-3657d3d6b5b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:49.033732 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed7d9f-226c-4d83-ad25-3657d3d6b5b8 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:49.044024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1100/2174] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:37:49.059022 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7decbf6-6037-4af8-aeef-1592a3406a97 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:49.063825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8301393-44e1-41c4-ac0b-ce193b384aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:49.064783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8301393-44e1-41c4-ac0b-ce193b384aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:49.065441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8301393-44e1-41c4-ac0b-ce193b384aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:49.065688 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8301393-44e1-41c4-ac0b-ce193b384aef admin admin] PATH_INFO: `/v3/users/d55c898180bd4cbf910f3d92045e7d67` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:49.067206 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8301393-44e1-41c4-ac0b-ce193b384aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d55c898180bd4cbf910f3d92045e7d67)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:49.069131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8301393-44e1-41c4-ac0b-ce193b384aef admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:49.120826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f8301393-44e1-41c4-ac0b-ce193b384aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd55c898180bd4cbf910f3d92045e7d67'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:49.121621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f8301393-44e1-41c4-ac0b-ce193b384aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd55c898180bd4cbf910f3d92045e7d67'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:49.140142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f8301393-44e1-41c4-ac0b-ce193b384aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd55c898180bd4cbf910f3d92045e7d67'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:49.141166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f8301393-44e1-41c4-ac0b-ce193b384aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd55c898180bd4cbf910f3d92045e7d67'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:49.152410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-f8301393-44e1-41c4-ac0b-ce193b384aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd55c898180bd4cbf910f3d92045e7d67'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:49.170419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1075/2175] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:48 2022] DELETE /identity/v3/users/d55c898180bd4cbf910f3d92045e7d67 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:37:49.360555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed7d9f-226c-4d83-ad25-3657d3d6b5b8 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:49.373323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed7d9f-226c-4d83-ad25-3657d3d6b5b8 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:49.409696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:49.439724 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:49.440705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:49.440972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:49.441249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] PATH_INFO: `/v3/projects/ed50fef9e1824a2c918d9649f226364d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:49.442482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed50fef9e1824a2c918d9649f226364d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:49.444392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:49.495723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed50fef9e1824a2c918d9649f226364d', 'request_id': 'req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:49.502426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed50fef9e1824a2c918d9649f226364d', 'request_id': 'req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:49.508406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed50fef9e1824a2c918d9649f226364d', 'request_id': 'req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:49.522313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] The token cache is being invalidate because project ed50fef9e1824a2c918d9649f226364d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:49.522596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:49.523234 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1a189e01-7a1a-4c9b-ba24-10fdb5529ff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:49.524818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1101/2176] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:49 2022] DELETE /identity/v3/projects/ed50fef9e1824a2c918d9649f226364d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:49.531197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb85119-79fd-4c71-be2a-69d71a1734a5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:49.531490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb85119-79fd-4c71-be2a-69d71a1734a5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:49.531775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb85119-79fd-4c71-be2a-69d71a1734a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:49.601054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1eb85119-79fd-4c71-be2a-69d71a1734a5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:49.602407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1eb85119-79fd-4c71-be2a-69d71a1734a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:49.661979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:49.687721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1076/2177] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:37:49.688996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11a214-c5b5-4ea3-8006-6f7d74f74bea None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:49.689355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11a214-c5b5-4ea3-8006-6f7d74f74bea None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:49.689709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11a214-c5b5-4ea3-8006-6f7d74f74bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:49.708693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0c11a214-c5b5-4ea3-8006-6f7d74f74bea None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:49.709886 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c11a214-c5b5-4ea3-8006-6f7d74f74bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:49.718067 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1102/2178] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:49.722675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb85119-79fd-4c71-be2a-69d71a1734a5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:49.732722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb85119-79fd-4c71-be2a-69d71a1734a5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:49.737394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86295497-91c4-4600-8865-1a2ba961f50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:49.738066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86295497-91c4-4600-8865-1a2ba961f50a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:49.738354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86295497-91c4-4600-8865-1a2ba961f50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:49.738634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86295497-91c4-4600-8865-1a2ba961f50a admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:49.739430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86295497-91c4-4600-8865-1a2ba961f50a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:37:49.739750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86295497-91c4-4600-8865-1a2ba961f50a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:49.741362 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86295497-91c4-4600-8865-1a2ba961f50a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:49.758057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1077/2179] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:37:49.759600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11a214-c5b5-4ea3-8006-6f7d74f74bea None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:49.769646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11a214-c5b5-4ea3-8006-6f7d74f74bea None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:49.774521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a87609-a13b-47bf-8459-3a2b99ea485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:49.775219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a87609-a13b-47bf-8459-3a2b99ea485c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:49.775505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a87609-a13b-47bf-8459-3a2b99ea485c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:49.775784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a87609-a13b-47bf-8459-3a2b99ea485c admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:49.776522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a87609-a13b-47bf-8459-3a2b99ea485c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:49.778807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a87609-a13b-47bf-8459-3a2b99ea485c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:49.810127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1103/2180] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:37:49 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:37:49.811262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86295497-91c4-4600-8865-1a2ba961f50a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:49.821965 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86295497-91c4-4600-8865-1a2ba961f50a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:49.887507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f5942-9d7e-4dd6-9a4a-91f9d92892bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:49.888213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f5942-9d7e-4dd6-9a4a-91f9d92892bd admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:49.888435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f5942-9d7e-4dd6-9a4a-91f9d92892bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:49.888713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f5942-9d7e-4dd6-9a4a-91f9d92892bd admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:49.889413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f5942-9d7e-4dd6-9a4a-91f9d92892bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:49.891408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f5942-9d7e-4dd6-9a4a-91f9d92892bd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:49.914449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1078/2181] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:37:49 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:37:49.918910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a87609-a13b-47bf-8459-3a2b99ea485c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:49.930101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a87609-a13b-47bf-8459-3a2b99ea485c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:49.935446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386811ea-5d4a-43fb-a118-303402c933a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:49.936200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386811ea-5d4a-43fb-a118-303402c933a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:49.936434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386811ea-5d4a-43fb-a118-303402c933a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:49.936747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386811ea-5d4a-43fb-a118-303402c933a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:49.937457 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386811ea-5d4a-43fb-a118-303402c933a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:49.939137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386811ea-5d4a-43fb-a118-303402c933a4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:49.950766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1104/2182] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:49.955160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f5942-9d7e-4dd6-9a4a-91f9d92892bd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:49.966935 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f5942-9d7e-4dd6-9a4a-91f9d92892bd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:49.972231 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d21ab-bcaa-401b-8e34-363b700d2dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:49.972914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d21ab-bcaa-401b-8e34-363b700d2dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:49.973201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d21ab-bcaa-401b-8e34-363b700d2dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:49.973476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d21ab-bcaa-401b-8e34-363b700d2dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:49.974227 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d21ab-bcaa-401b-8e34-363b700d2dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:49.975890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57d21ab-bcaa-401b-8e34-363b700d2dd9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:49.986422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1079/2183] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:37:49.990823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386811ea-5d4a-43fb-a118-303402c933a4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:50.001017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386811ea-5d4a-43fb-a118-303402c933a4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:50.006432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15b35d-e2b9-4fa6-838c-3a2110e68b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:50.007516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15b35d-e2b9-4fa6-838c-3a2110e68b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:50.007834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15b35d-e2b9-4fa6-838c-3a2110e68b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:50.008106 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15b35d-e2b9-4fa6-838c-3a2110e68b84 admin admin] PATH_INFO: `/v3/projects/13433870415b42eaaa81dfebcb1831bc/users/d032189e5a7c41ee8d40f6572b78951a/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:50.025142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15b35d-e2b9-4fa6-838c-3a2110e68b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13433870415b42eaaa81dfebcb1831bc, user_id=d032189e5a7c41ee8d40f6572b78951a, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:50.027829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15b35d-e2b9-4fa6-838c-3a2110e68b84 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:50.043562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1105/2184] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:37:49 2022] PUT /identity/v3/projects/13433870415b42eaaa81dfebcb1831bc/users/d032189e5a7c41ee8d40f6572b78951a/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:50.054089 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4c71c-79ce-4da4-a966-d3a339d1bf66 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:50.054376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4c71c-79ce-4da4-a966-d3a339d1bf66 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:50.054656 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4c71c-79ce-4da4-a966-d3a339d1bf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:50.084569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e0c4c71c-79ce-4da4-a966-d3a339d1bf66 None None] MFA Rules not processed for user `d032189e5a7c41ee8d40f6572b78951a`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:50.086727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e0c4c71c-79ce-4da4-a966-d3a339d1bf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:50.159741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1080/2185] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:50.178936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15b35d-e2b9-4fa6-838c-3a2110e68b84 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:50.180932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5e15b35d-e2b9-4fa6-838c-3a2110e68b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:50.243904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:50.263468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15b35d-e2b9-4fa6-838c-3a2110e68b84 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:50.269699 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad93a70-aa7b-46d1-8a06-d1c410ab0d8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:50.270366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad93a70-aa7b-46d1-8a06-d1c410ab0d8a service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:50.270648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad93a70-aa7b-46d1-8a06-d1c410ab0d8a service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:50.270962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad93a70-aa7b-46d1-8a06-d1c410ab0d8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:50.280939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad93a70-aa7b-46d1-8a06-d1c410ab0d8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:50.282519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad93a70-aa7b-46d1-8a06-d1c410ab0d8a service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:50.292586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1106/2186] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:58.070601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4c71c-79ce-4da4-a966-d3a339d1bf66 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:58.072336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e0c4c71c-79ce-4da4-a966-d3a339d1bf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:58.126917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:37:58.143360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4c71c-79ce-4da4-a966-d3a339d1bf66 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:58.148120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:58.148999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:58.149279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:58.149616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] PATH_INFO: `/v3/users/ac61282884a54556bb611c59c30d75a2` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:58.150843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac61282884a54556bb611c59c30d75a2)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:58.152766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:58.203982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac61282884a54556bb611c59c30d75a2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:58.204649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac61282884a54556bb611c59c30d75a2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:58.217221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac61282884a54556bb611c59c30d75a2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:58.217886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac61282884a54556bb611c59c30d75a2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:58.227071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac61282884a54556bb611c59c30d75a2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:58.239969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1081/2187] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:37:58 2022] DELETE /identity/v3/users/ac61282884a54556bb611c59c30d75a2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:37:58.244937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d581b-d9b0-4c27-9cb7-90f64ec5eb3d None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:58.245449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d581b-d9b0-4c27-9cb7-90f64ec5eb3d None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:58.245543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d581b-d9b0-4c27-9cb7-90f64ec5eb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:58.263618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-920d581b-d9b0-4c27-9cb7-90f64ec5eb3d None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:37:58.264816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-920d581b-d9b0-4c27-9cb7-90f64ec5eb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:58.333564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1107/2188] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:37:58.345414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:58.347049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:58.441420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3d5b3-4a58-4657-b5dd-2f43a054a0d2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:58.445473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5a549-781f-4ffb-ab8b-4b20cf194de2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:58.446127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5a549-781f-4ffb-ab8b-4b20cf194de2 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:58.446912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5a549-781f-4ffb-ab8b-4b20cf194de2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:58.447248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a5a549-781f-4ffb-ab8b-4b20cf194de2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:58.457347 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a5a549-781f-4ffb-ab8b-4b20cf194de2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:58.458851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a5a549-781f-4ffb-ab8b-4b20cf194de2 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:58.469943 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1082/2189] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:37:58.653026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d581b-d9b0-4c27-9cb7-90f64ec5eb3d None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:37:58.657507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-920d581b-d9b0-4c27-9cb7-90f64ec5eb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:37:58.669969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d581b-d9b0-4c27-9cb7-90f64ec5eb3d None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:37:58.675426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:37:58.676307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:37:58.676588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:37:58.676860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] PATH_INFO: `/v3/projects/3faae8ed99784a52b784c22e77de9890` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:37:58.678142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3faae8ed99784a52b784c22e77de9890)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:37:58.680087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:37:58.731217 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3faae8ed99784a52b784c22e77de9890', 'request_id': 'req-6691428a-d9f3-49f5-a424-6d5644ea9f8a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:58.737775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3faae8ed99784a52b784c22e77de9890', 'request_id': 'req-6691428a-d9f3-49f5-a424-6d5644ea9f8a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:58.747077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3faae8ed99784a52b784c22e77de9890', 'request_id': 'req-6691428a-d9f3-49f5-a424-6d5644ea9f8a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:58.762422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] The token cache is being invalidate because project 3faae8ed99784a52b784c22e77de9890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:37:58.762745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:58.763373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6691428a-d9f3-49f5-a424-6d5644ea9f8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:37:58.765260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1108/2190] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:37:58 2022] DELETE /identity/v3/projects/3faae8ed99784a52b784c22e77de9890 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:38:07.013258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5a549-781f-4ffb-ab8b-4b20cf194de2 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:07.014863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-61a5a549-781f-4ffb-ab8b-4b20cf194de2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:07.028665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a5a549-781f-4ffb-ab8b-4b20cf194de2 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:07.077825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:07.087854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:07.088801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:07.089093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:07.089451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] PATH_INFO: `/v3/users/86de1d29eeed434f9aebdefe57e25c47` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:07.091483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=86de1d29eeed434f9aebdefe57e25c47)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:07.093618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:07.154173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86de1d29eeed434f9aebdefe57e25c47'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:07.155330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86de1d29eeed434f9aebdefe57e25c47'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:07.174480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86de1d29eeed434f9aebdefe57e25c47'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:07.175314 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86de1d29eeed434f9aebdefe57e25c47'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:07.184058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86de1d29eeed434f9aebdefe57e25c47'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:07.197225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1083/2191] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:07 2022] DELETE /identity/v3/users/86de1d29eeed434f9aebdefe57e25c47 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:07.202377 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b8524-6b1c-4795-87d6-7d9885e62549 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:07.202758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b8524-6b1c-4795-87d6-7d9885e62549 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:07.202758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b8524-6b1c-4795-87d6-7d9885e62549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:07.224190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f98b8524-6b1c-4795-87d6-7d9885e62549 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:07.225435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f98b8524-6b1c-4795-87d6-7d9885e62549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:07.313371 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1109/2192] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:07.323440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:07.324884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:07.382045 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ee2be4-28f1-4839-8b7f-b3378b3e8c4c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:07.385843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e700d1-8868-449a-b1d7-0c4ca1827c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:07.386499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e700d1-8868-449a-b1d7-0c4ca1827c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:07.386820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e700d1-8868-449a-b1d7-0c4ca1827c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:07.387102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e700d1-8868-449a-b1d7-0c4ca1827c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:07.396582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e700d1-8868-449a-b1d7-0c4ca1827c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:07.398020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e700d1-8868-449a-b1d7-0c4ca1827c84 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:07.407732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1084/2193] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:07.690736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b8524-6b1c-4795-87d6-7d9885e62549 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:07.692301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f98b8524-6b1c-4795-87d6-7d9885e62549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:07.704941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b8524-6b1c-4795-87d6-7d9885e62549 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:07.710191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:07.711244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:07.711525 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:07.711799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] PATH_INFO: `/v3/projects/467375e5e3e447f1b73f5b26bd81ec71` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:07.714249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] RBAC: Authorizing `identity:delete_project(project_id=467375e5e3e447f1b73f5b26bd81ec71)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:07.716585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:07.719649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:07.774223 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '467375e5e3e447f1b73f5b26bd81ec71', 'request_id': 'req-c89c85ce-b75e-4096-9504-0e84b975776b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:07.783133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '467375e5e3e447f1b73f5b26bd81ec71', 'request_id': 'req-c89c85ce-b75e-4096-9504-0e84b975776b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:07.789282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '467375e5e3e447f1b73f5b26bd81ec71', 'request_id': 'req-c89c85ce-b75e-4096-9504-0e84b975776b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:07.803087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] The token cache is being invalidate because project 467375e5e3e447f1b73f5b26bd81ec71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:07.803374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:07.804016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c89c85ce-b75e-4096-9504-0e84b975776b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:07.805648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1110/2194] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:07 2022] DELETE /identity/v3/projects/467375e5e3e447f1b73f5b26bd81ec71 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:17.403431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:17.800029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=23,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:23.094458 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e700d1-8868-449a-b1d7-0c4ca1827c84 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:23.095866 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d0e700d1-8868-449a-b1d7-0c4ca1827c84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:23.110643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e700d1-8868-449a-b1d7-0c4ca1827c84 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:23.171383 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:23.172261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:23.172548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:23.172825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] PATH_INFO: `/v3/users/d032189e5a7c41ee8d40f6572b78951a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:23.174088 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d032189e5a7c41ee8d40f6572b78951a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:23.176481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:23.228910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd032189e5a7c41ee8d40f6572b78951a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:23.229671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd032189e5a7c41ee8d40f6572b78951a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:23.243954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd032189e5a7c41ee8d40f6572b78951a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:23.244857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd032189e5a7c41ee8d40f6572b78951a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:23.256745 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd032189e5a7c41ee8d40f6572b78951a'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:23.271514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1085/2195] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:23 2022] DELETE /identity/v3/users/d032189e5a7c41ee8d40f6572b78951a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:23.276221 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f9f18-b522-4ddd-8e79-1b0c99dd2227 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:23.276747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f9f18-b522-4ddd-8e79-1b0c99dd2227 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:23.276747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f9f18-b522-4ddd-8e79-1b0c99dd2227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:23.296117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-e53f9f18-b522-4ddd-8e79-1b0c99dd2227 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:23.297344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e53f9f18-b522-4ddd-8e79-1b0c99dd2227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:23.358379 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1111/2196] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:23.368822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:23.370498 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:23.431218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b01e6e-8fb6-4d82-a7be-cd0bcd455638 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:23.434959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226cdde-c18f-4278-b829-e80e05242121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:23.435609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226cdde-c18f-4278-b829-e80e05242121 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:23.435893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226cdde-c18f-4278-b829-e80e05242121 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:23.436172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226cdde-c18f-4278-b829-e80e05242121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:23.445449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226cdde-c18f-4278-b829-e80e05242121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:23.446922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c226cdde-c18f-4278-b829-e80e05242121 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:23.456960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1086/2197] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:23.737588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f9f18-b522-4ddd-8e79-1b0c99dd2227 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:23.739241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e53f9f18-b522-4ddd-8e79-1b0c99dd2227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:23.754354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f9f18-b522-4ddd-8e79-1b0c99dd2227 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:23.759670 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:23.760594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:23.760862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:23.761144 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] PATH_INFO: `/v3/projects/13433870415b42eaaa81dfebcb1831bc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:23.762352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13433870415b42eaaa81dfebcb1831bc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:23.764389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:23.816788 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13433870415b42eaaa81dfebcb1831bc', 'request_id': 'req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:23.823125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13433870415b42eaaa81dfebcb1831bc', 'request_id': 'req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:23.831241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13433870415b42eaaa81dfebcb1831bc', 'request_id': 'req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:23.847527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] The token cache is being invalidate because project 13433870415b42eaaa81dfebcb1831bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:23.847813 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:23.848536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f9f805c2-bbf6-4509-9e9f-7cfdd45ceb68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:23.850303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1112/2198] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:23 2022] DELETE /identity/v3/projects/13433870415b42eaaa81dfebcb1831bc => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:23.855490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e555b76-57ea-4eb5-af4f-2d8c72417320 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:23.855854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e555b76-57ea-4eb5-af4f-2d8c72417320 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:23.856196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e555b76-57ea-4eb5-af4f-2d8c72417320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:23.885975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8e555b76-57ea-4eb5-af4f-2d8c72417320 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:23.887218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8e555b76-57ea-4eb5-af4f-2d8c72417320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:23.898094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:23.953329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1087/2199] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:23.961332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f1a08-5e65-4c42-8fac-937703936f97 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:23.961617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f1a08-5e65-4c42-8fac-937703936f97 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:23.961891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f1a08-5e65-4c42-8fac-937703936f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:23.980735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-770f1a08-5e65-4c42-8fac-937703936f97 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:23.981938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-770f1a08-5e65-4c42-8fac-937703936f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:23.990648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1113/2200] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:23.996506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e555b76-57ea-4eb5-af4f-2d8c72417320 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:24.006586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e555b76-57ea-4eb5-af4f-2d8c72417320 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:24.013041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08760b1c-d748-48e6-b2e8-3a6871b147cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:24.013674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08760b1c-d748-48e6-b2e8-3a6871b147cf admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:24.013954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08760b1c-d748-48e6-b2e8-3a6871b147cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:24.014225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08760b1c-d748-48e6-b2e8-3a6871b147cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:24.014995 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08760b1c-d748-48e6-b2e8-3a6871b147cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:38:24.015302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08760b1c-d748-48e6-b2e8-3a6871b147cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:24.016932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08760b1c-d748-48e6-b2e8-3a6871b147cf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:24.028994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1088/2201] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:38:24.033421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f1a08-5e65-4c42-8fac-937703936f97 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:24.042991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f1a08-5e65-4c42-8fac-937703936f97 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:24.048681 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fc8b4-6d28-4245-987f-6ced2e8a8c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:24.049392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fc8b4-6d28-4245-987f-6ced2e8a8c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:24.049674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fc8b4-6d28-4245-987f-6ced2e8a8c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:24.049947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fc8b4-6d28-4245-987f-6ced2e8a8c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:24.050646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fc8b4-6d28-4245-987f-6ced2e8a8c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:24.052603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fc8b4-6d28-4245-987f-6ced2e8a8c4b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:24.081816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1114/2202] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:38:24 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:38:24.085829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08760b1c-d748-48e6-b2e8-3a6871b147cf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:24.097368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08760b1c-d748-48e6-b2e8-3a6871b147cf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:24.158112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38073e-c437-4eb6-ae22-3be81c1a479f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:24.158850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38073e-c437-4eb6-ae22-3be81c1a479f admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:24.159130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38073e-c437-4eb6-ae22-3be81c1a479f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:24.159402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38073e-c437-4eb6-ae22-3be81c1a479f admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:24.160107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38073e-c437-4eb6-ae22-3be81c1a479f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:24.162018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38073e-c437-4eb6-ae22-3be81c1a479f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:24.182675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1089/2203] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:38:24 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:38:24.187125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fc8b4-6d28-4245-987f-6ced2e8a8c4b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:24.197995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fc8b4-6d28-4245-987f-6ced2e8a8c4b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:24.203677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4315e9-5514-45ec-b6fb-35e1af725eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:24.204386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4315e9-5514-45ec-b6fb-35e1af725eae admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:24.204669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4315e9-5514-45ec-b6fb-35e1af725eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:24.204946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4315e9-5514-45ec-b6fb-35e1af725eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:24.205771 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4315e9-5514-45ec-b6fb-35e1af725eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:24.207493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4315e9-5514-45ec-b6fb-35e1af725eae admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:24.220427 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1115/2204] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:38:24.224780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38073e-c437-4eb6-ae22-3be81c1a479f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:24.235299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38073e-c437-4eb6-ae22-3be81c1a479f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:24.240883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b4911-004a-4dd7-940b-e6667b948bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:24.241545 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b4911-004a-4dd7-940b-e6667b948bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:24.241825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b4911-004a-4dd7-940b-e6667b948bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:24.242096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b4911-004a-4dd7-940b-e6667b948bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:24.242866 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b4911-004a-4dd7-940b-e6667b948bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:24.244722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b4911-004a-4dd7-940b-e6667b948bd2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:24.255389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1090/2205] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:38:24.259583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4315e9-5514-45ec-b6fb-35e1af725eae admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:24.270124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4315e9-5514-45ec-b6fb-35e1af725eae admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:24.274604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244d55d-9764-4fce-a235-d5c1ae476aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:24.275521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244d55d-9764-4fce-a235-d5c1ae476aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:24.275805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244d55d-9764-4fce-a235-d5c1ae476aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:24.276077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244d55d-9764-4fce-a235-d5c1ae476aa0 admin admin] PATH_INFO: `/v3/projects/e6f0c6116bb3466d9a458cc2086a9e72/users/d02d5f423a734c58868d139e0f5250b5/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:24.291436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244d55d-9764-4fce-a235-d5c1ae476aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6f0c6116bb3466d9a458cc2086a9e72, user_id=d02d5f423a734c58868d139e0f5250b5, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:24.294076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4244d55d-9764-4fce-a235-d5c1ae476aa0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:24.308059 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1116/2206] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:24 2022] PUT /identity/v3/projects/e6f0c6116bb3466d9a458cc2086a9e72/users/d02d5f423a734c58868d139e0f5250b5/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:24.318987 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdcf07-23c3-4909-8853-664cd29e698d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:24.319286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdcf07-23c3-4909-8853-664cd29e698d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:24.319556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdcf07-23c3-4909-8853-664cd29e698d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:24.347906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f0bdcf07-23c3-4909-8853-664cd29e698d None None] MFA Rules not processed for user `d02d5f423a734c58868d139e0f5250b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:24.349123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f0bdcf07-23c3-4909-8853-664cd29e698d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:24.421290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1091/2207] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:24.432761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244d55d-9764-4fce-a235-d5c1ae476aa0 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:24.434535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4244d55d-9764-4fce-a235-d5c1ae476aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:24.465438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:24.541413 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244d55d-9764-4fce-a235-d5c1ae476aa0 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:24.545273 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be3b11-528b-46fe-bb28-89014837dc1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:24.545913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3b11-528b-46fe-bb28-89014837dc1b service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:24.546189 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3b11-528b-46fe-bb28-89014837dc1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:24.546434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3b11-528b-46fe-bb28-89014837dc1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:24.555034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be3b11-528b-46fe-bb28-89014837dc1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:24.556463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be3b11-528b-46fe-bb28-89014837dc1b service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:24.565558 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1117/2208] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:34.418822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:34.561609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=26 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:39.456321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eee49a-0222-41c9-9817-398888104cd9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:39.456321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eee49a-0222-41c9-9817-398888104cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:39.456321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eee49a-0222-41c9-9817-398888104cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:39.474178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a3eee49a-0222-41c9-9817-398888104cd9 None None] MFA Rules not processed for user `ac984d1f6efa403e85483bd7fa78000e`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:39.475436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a3eee49a-0222-41c9-9817-398888104cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:39.536279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1092/2209] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:43.762439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be3b11-528b-46fe-bb28-89014837dc1b service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:43.763930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f9be3b11-528b-46fe-bb28-89014837dc1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:43.873380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be3b11-528b-46fe-bb28-89014837dc1b service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:43.878845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576457f-3b7c-4f74-98b4-404b3e90403a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:43.879503 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576457f-3b7c-4f74-98b4-404b3e90403a service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:43.879782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576457f-3b7c-4f74-98b4-404b3e90403a service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:43.880080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576457f-3b7c-4f74-98b4-404b3e90403a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:43.882176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0576457f-3b7c-4f74-98b4-404b3e90403a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:43.893970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576457f-3b7c-4f74-98b4-404b3e90403a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:43.895674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576457f-3b7c-4f74-98b4-404b3e90403a service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:43.905977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1118/2210] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:49.153378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eee49a-0222-41c9-9817-398888104cd9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:49.225405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eee49a-0222-41c9-9817-398888104cd9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:49.230397 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:49.231246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:49.231529 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:49.231806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] PATH_INFO: `/v3/users/d02d5f423a734c58868d139e0f5250b5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:49.233077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02d5f423a734c58868d139e0f5250b5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:49.235048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:49.288841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02d5f423a734c58868d139e0f5250b5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.289571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02d5f423a734c58868d139e0f5250b5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.308270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02d5f423a734c58868d139e0f5250b5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.309033 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02d5f423a734c58868d139e0f5250b5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.320902 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02d5f423a734c58868d139e0f5250b5'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.342196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1093/2211] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:49 2022] DELETE /identity/v3/users/d02d5f423a734c58868d139e0f5250b5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:49.344115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bfb38-89af-4c58-8323-3add08af4fcb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:49.344764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bfb38-89af-4c58-8323-3add08af4fcb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:49.344764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9bfb38-89af-4c58-8323-3add08af4fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:49.364147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5a9bfb38-89af-4c58-8323-3add08af4fcb None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:49.365428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5a9bfb38-89af-4c58-8323-3add08af4fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:49.428765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1119/2212] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:49.442981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:49.444630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:49.484736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:49.516057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3593cc4-357d-46f3-801c-bbfcfbf9495d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:49.519764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ec1f4-6c7f-495d-8e59-21bdef4eedce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:49.520416 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ec1f4-6c7f-495d-8e59-21bdef4eedce service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:49.520701 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ec1f4-6c7f-495d-8e59-21bdef4eedce service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:49.520979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ec1f4-6c7f-495d-8e59-21bdef4eedce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:49.530147 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ec1f4-6c7f-495d-8e59-21bdef4eedce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:49.531653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ec1f4-6c7f-495d-8e59-21bdef4eedce service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:49.544913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1094/2213] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:49.807808 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bfb38-89af-4c58-8323-3add08af4fcb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:49.809745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5a9bfb38-89af-4c58-8323-3add08af4fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:49.825350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9bfb38-89af-4c58-8323-3add08af4fcb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:49.831853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdfe68-6b20-453e-a442-99d98b8977f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:49.833663 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdfe68-6b20-453e-a442-99d98b8977f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:49.833663 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdfe68-6b20-453e-a442-99d98b8977f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:49.833663 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdfe68-6b20-453e-a442-99d98b8977f3 admin admin] PATH_INFO: `/v3/users/258f273e20a44115ab9b9db0fffde8ce` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:49.835076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdfe68-6b20-453e-a442-99d98b8977f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=258f273e20a44115ab9b9db0fffde8ce)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:49.837121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdfe68-6b20-453e-a442-99d98b8977f3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:49.847414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ec1f4-6c7f-495d-8e59-21bdef4eedce service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:49.854747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0d1ec1f4-6c7f-495d-8e59-21bdef4eedce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:49.884384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ec1f4-6c7f-495d-8e59-21bdef4eedce service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:49.890389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:49.891856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:49.892148 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:49.892434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] PATH_INFO: `/v3/projects/e6f0c6116bb3466d9a458cc2086a9e72` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:49.894453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6f0c6116bb3466d9a458cc2086a9e72)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:49.896432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:49.902880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-87bdfe68-6b20-453e-a442-99d98b8977f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '258f273e20a44115ab9b9db0fffde8ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.903469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-87bdfe68-6b20-453e-a442-99d98b8977f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '258f273e20a44115ab9b9db0fffde8ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.918743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-87bdfe68-6b20-453e-a442-99d98b8977f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '258f273e20a44115ab9b9db0fffde8ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.920339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-87bdfe68-6b20-453e-a442-99d98b8977f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '258f273e20a44115ab9b9db0fffde8ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.922415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:49.932345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-87bdfe68-6b20-453e-a442-99d98b8977f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '258f273e20a44115ab9b9db0fffde8ce'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.953656 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1120/2214] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:49 2022] DELETE /identity/v3/users/258f273e20a44115ab9b9db0fffde8ce => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:49.958543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae78ec-cc6f-41fc-a55e-8c2ac8b34102 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:49.958923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae78ec-cc6f-41fc-a55e-8c2ac8b34102 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:49.959394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae78ec-cc6f-41fc-a55e-8c2ac8b34102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:49.963992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6f0c6116bb3466d9a458cc2086a9e72', 'request_id': 'req-60919ca2-d94b-4c45-96a3-080ad5011b26'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.971820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6f0c6116bb3466d9a458cc2086a9e72', 'request_id': 'req-60919ca2-d94b-4c45-96a3-080ad5011b26'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.978563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6f0c6116bb3466d9a458cc2086a9e72', 'request_id': 'req-60919ca2-d94b-4c45-96a3-080ad5011b26'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.980853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5fae78ec-cc6f-41fc-a55e-8c2ac8b34102 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:49.983239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5fae78ec-cc6f-41fc-a55e-8c2ac8b34102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:49.997252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] The token cache is being invalidate because project e6f0c6116bb3466d9a458cc2086a9e72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:49.997632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:49.998325 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-60919ca2-d94b-4c45-96a3-080ad5011b26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:50.000030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1095/2215] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:49 2022] DELETE /identity/v3/projects/e6f0c6116bb3466d9a458cc2086a9e72 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:38:50.005961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258dd5c-e866-44fb-bdc9-22036cd2b4e1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.006360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258dd5c-e866-44fb-bdc9-22036cd2b4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.006767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258dd5c-e866-44fb-bdc9-22036cd2b4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.027655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3258dd5c-e866-44fb-bdc9-22036cd2b4e1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:50.028868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3258dd5c-e866-44fb-bdc9-22036cd2b4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:50.054832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1121/2216] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:50.063449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae78ec-cc6f-41fc-a55e-8c2ac8b34102 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.065049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5fae78ec-cc6f-41fc-a55e-8c2ac8b34102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:50.095072 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1096/2217] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:50.104868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad6b6f-f9e7-4be1-9ecf-814d4367dd6a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.105271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad6b6f-f9e7-4be1-9ecf-814d4367dd6a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.105612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad6b6f-f9e7-4be1-9ecf-814d4367dd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.126545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae78ec-cc6f-41fc-a55e-8c2ac8b34102 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.127425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b7ad6b6f-f9e7-4be1-9ecf-814d4367dd6a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:50.128805 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b7ad6b6f-f9e7-4be1-9ecf-814d4367dd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:50.132023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bbe22-f2d9-4d61-8c84-321df745ac1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.132688 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bbe22-f2d9-4d61-8c84-321df745ac1f service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.132969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bbe22-f2d9-4d61-8c84-321df745ac1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.133291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2bbe22-f2d9-4d61-8c84-321df745ac1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.138931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1097/2218] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:50.143094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad6b6f-f9e7-4be1-9ecf-814d4367dd6a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.143394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bbe22-f2d9-4d61-8c84-321df745ac1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.144956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2bbe22-f2d9-4d61-8c84-321df745ac1f service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.153125 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad6b6f-f9e7-4be1-9ecf-814d4367dd6a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.155021 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1122/2219] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:38:50.157731 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab3931-8295-488f-8a85-cd61f2a3c37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.158424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab3931-8295-488f-8a85-cd61f2a3c37c admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.158880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab3931-8295-488f-8a85-cd61f2a3c37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.159040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab3931-8295-488f-8a85-cd61f2a3c37c admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.161213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab3931-8295-488f-8a85-cd61f2a3c37c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:38:50.161213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab3931-8295-488f-8a85-cd61f2a3c37c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.161818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab3931-8295-488f-8a85-cd61f2a3c37c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.175267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1098/2220] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:38:50.179706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bbe22-f2d9-4d61-8c84-321df745ac1f service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.190157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2bbe22-f2d9-4d61-8c84-321df745ac1f service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.195706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf9a25-67f3-4d24-a200-dd2e038797d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.196353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf9a25-67f3-4d24-a200-dd2e038797d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.196640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf9a25-67f3-4d24-a200-dd2e038797d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.196909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf9a25-67f3-4d24-a200-dd2e038797d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.197625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf9a25-67f3-4d24-a200-dd2e038797d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.199612 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf9a25-67f3-4d24-a200-dd2e038797d4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.232430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1123/2221] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:38:50 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:38:50.233791 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab3931-8295-488f-8a85-cd61f2a3c37c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.244498 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab3931-8295-488f-8a85-cd61f2a3c37c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.275193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:50.308759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bf4df-7b71-4a3e-8da4-38c4f1222903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.309450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf4df-7b71-4a3e-8da4-38c4f1222903 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.309737 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf4df-7b71-4a3e-8da4-38c4f1222903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.310021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bf4df-7b71-4a3e-8da4-38c4f1222903 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.310754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bf4df-7b71-4a3e-8da4-38c4f1222903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.312684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bf4df-7b71-4a3e-8da4-38c4f1222903 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.336195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1099/2222] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:38:50 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:38:50.340656 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf9a25-67f3-4d24-a200-dd2e038797d4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.350895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf9a25-67f3-4d24-a200-dd2e038797d4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.355861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bcbad-0e87-44e0-87ed-5984cf15ba26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.356538 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bcbad-0e87-44e0-87ed-5984cf15ba26 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.356820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bcbad-0e87-44e0-87ed-5984cf15ba26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.357098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5bcbad-0e87-44e0-87ed-5984cf15ba26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.357832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bcbad-0e87-44e0-87ed-5984cf15ba26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.359674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5bcbad-0e87-44e0-87ed-5984cf15ba26 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.370997 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1124/2223] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:38:50.374663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bf4df-7b71-4a3e-8da4-38c4f1222903 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.385609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bf4df-7b71-4a3e-8da4-38c4f1222903 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.390456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71c95f-4aaa-4e0d-8be3-85d1b5a57686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.391169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71c95f-4aaa-4e0d-8be3-85d1b5a57686 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.391490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71c95f-4aaa-4e0d-8be3-85d1b5a57686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.391735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71c95f-4aaa-4e0d-8be3-85d1b5a57686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.392471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71c95f-4aaa-4e0d-8be3-85d1b5a57686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.394078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d71c95f-4aaa-4e0d-8be3-85d1b5a57686 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.406740 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1100/2224] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:38:50.409820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bcbad-0e87-44e0-87ed-5984cf15ba26 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.421411 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5bcbad-0e87-44e0-87ed-5984cf15ba26 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.427891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d6778-a12c-4bce-b02b-85382aa3828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.428809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d6778-a12c-4bce-b02b-85382aa3828f admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.429206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d6778-a12c-4bce-b02b-85382aa3828f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.429485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d6778-a12c-4bce-b02b-85382aa3828f admin admin] PATH_INFO: `/v3/projects/033eb927966e44668beb535cd0a23b91/users/c61d9bf0b11449d4a60859170d9d570f/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.446507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71c95f-4aaa-4e0d-8be3-85d1b5a57686 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.449082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7d71c95f-4aaa-4e0d-8be3-85d1b5a57686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:50.450304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d6778-a12c-4bce-b02b-85382aa3828f admin admin] RBAC: Authorizing `identity:create_grant(project_id=033eb927966e44668beb535cd0a23b91, user_id=c61d9bf0b11449d4a60859170d9d570f, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.459881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d6778-a12c-4bce-b02b-85382aa3828f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.481144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d71c95f-4aaa-4e0d-8be3-85d1b5a57686 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.489033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1125/2225] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:50 2022] PUT /identity/v3/projects/033eb927966e44668beb535cd0a23b91/users/c61d9bf0b11449d4a60859170d9d570f/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:50.499374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1e005-8a47-4622-81de-c9e0b43a519a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.500105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1e005-8a47-4622-81de-c9e0b43a519a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.500105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1e005-8a47-4622-81de-c9e0b43a519a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.532046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-26e1e005-8a47-4622-81de-c9e0b43a519a None None] MFA Rules not processed for user `c61d9bf0b11449d4a60859170d9d570f`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:50.533336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-26e1e005-8a47-4622-81de-c9e0b43a519a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:50.550991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:50.565641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.566546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.566856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.567131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] PATH_INFO: `/v3/projects/a4675925d46d42bfa23221229a0b85aa` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.568525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4675925d46d42bfa23221229a0b85aa)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.570520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.607527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1126/2226] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:50.612450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1e005-8a47-4622-81de-c9e0b43a519a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.623401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1e005-8a47-4622-81de-c9e0b43a519a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.630762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4675925d46d42bfa23221229a0b85aa', 'request_id': 'req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:50.637912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4675925d46d42bfa23221229a0b85aa', 'request_id': 'req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:50.643879 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4675925d46d42bfa23221229a0b85aa', 'request_id': 'req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:50.659557 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] The token cache is being invalidate because project a4675925d46d42bfa23221229a0b85aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:50.659557 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:50.660286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ab42a9f5-e5a3-4ecb-8b2c-145fca918b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:50.662009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1101/2227] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:50 2022] DELETE /identity/v3/projects/a4675925d46d42bfa23221229a0b85aa => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:38:50.667532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5656684-c57a-45a2-bda2-4f968a77f259 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.667899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5656684-c57a-45a2-bda2-4f968a77f259 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.668260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5656684-c57a-45a2-bda2-4f968a77f259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.686552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b5656684-c57a-45a2-bda2-4f968a77f259 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:50.687838 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b5656684-c57a-45a2-bda2-4f968a77f259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:50.700427 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8379756-aa87-4c6d-ae24-c92f2a085a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.701337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379756-aa87-4c6d-ae24-c92f2a085a69 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.702027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379756-aa87-4c6d-ae24-c92f2a085a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.702027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8379756-aa87-4c6d-ae24-c92f2a085a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.702590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8379756-aa87-4c6d-ae24-c92f2a085a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.704584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8379756-aa87-4c6d-ae24-c92f2a085a69 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.723007 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:50.745326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1127/2228] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:38:50 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:38:50.749304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8379756-aa87-4c6d-ae24-c92f2a085a69 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.750998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f8379756-aa87-4c6d-ae24-c92f2a085a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:50.764020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1102/2229] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:50.765748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3962c-3621-40d2-8ab4-0019c6dca1d6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.766139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3962c-3621-40d2-8ab4-0019c6dca1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.766515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3962c-3621-40d2-8ab4-0019c6dca1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.787917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c1e3962c-3621-40d2-8ab4-0019c6dca1d6 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:50.789437 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c1e3962c-3621-40d2-8ab4-0019c6dca1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:50.832658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8379756-aa87-4c6d-ae24-c92f2a085a69 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.837924 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f58e9-d00a-48f6-9e0c-77be0523cfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.838615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f58e9-d00a-48f6-9e0c-77be0523cfb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.839011 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f58e9-d00a-48f6-9e0c-77be0523cfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.839325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f58e9-d00a-48f6-9e0c-77be0523cfb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.840476 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f58e9-d00a-48f6-9e0c-77be0523cfb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.842447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f58e9-d00a-48f6-9e0c-77be0523cfb4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.865869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1103/2230] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:50.869325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1128/2231] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:38:50 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:38:50.873344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3962c-3621-40d2-8ab4-0019c6dca1d6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.873492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f58e9-d00a-48f6-9e0c-77be0523cfb4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.884365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3962c-3621-40d2-8ab4-0019c6dca1d6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.885797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f58e9-d00a-48f6-9e0c-77be0523cfb4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.890120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a89457-ef43-403d-b86b-ae1806502093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.891044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a89457-ef43-403d-b86b-ae1806502093 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.891500 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a89457-ef43-403d-b86b-ae1806502093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.892037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a89457-ef43-403d-b86b-ae1806502093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.892938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a89457-ef43-403d-b86b-ae1806502093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.893153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e4e9d-13ee-489d-8e2b-208806be0c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.893936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e4e9d-13ee-489d-8e2b-208806be0c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.894223 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e4e9d-13ee-489d-8e2b-208806be0c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.894498 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e4e9d-13ee-489d-8e2b-208806be0c07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.895284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e4e9d-13ee-489d-8e2b-208806be0c07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:38:50.895593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a89457-ef43-403d-b86b-ae1806502093 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.895707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e4e9d-13ee-489d-8e2b-208806be0c07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.897205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e4e9d-13ee-489d-8e2b-208806be0c07 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.907649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1104/2232] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:38:50.913631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a89457-ef43-403d-b86b-ae1806502093 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.915740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1129/2233] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:38:50.920263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e4e9d-13ee-489d-8e2b-208806be0c07 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.926169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a89457-ef43-403d-b86b-ae1806502093 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.930895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e4e9d-13ee-489d-8e2b-208806be0c07 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:50.934309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746cd38a-a428-46a6-a417-5cfdcb725603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.935372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cd38a-a428-46a6-a417-5cfdcb725603 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.935740 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cd38a-a428-46a6-a417-5cfdcb725603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.936054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746cd38a-a428-46a6-a417-5cfdcb725603 admin admin] PATH_INFO: `/v3/projects/e296389d57ef42be9db9e476747c23ea/users/32a605a91297415ab1782b2d71a9502f/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.938330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252c506-24e8-44be-a301-18f8bc2463dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:50.939297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252c506-24e8-44be-a301-18f8bc2463dd admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:50.939651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252c506-24e8-44be-a301-18f8bc2463dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:50.940881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252c506-24e8-44be-a301-18f8bc2463dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:50.941851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252c506-24e8-44be-a301-18f8bc2463dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.944302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252c506-24e8-44be-a301-18f8bc2463dd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.957046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746cd38a-a428-46a6-a417-5cfdcb725603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e296389d57ef42be9db9e476747c23ea, user_id=32a605a91297415ab1782b2d71a9502f, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:50.959761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746cd38a-a428-46a6-a417-5cfdcb725603 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:50.969452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:50.983918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1130/2234] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:38:50 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:38:50.985758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1105/2235] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:50 2022] PUT /identity/v3/projects/e296389d57ef42be9db9e476747c23ea/users/32a605a91297415ab1782b2d71a9502f/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:38:50.985900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252c506-24e8-44be-a301-18f8bc2463dd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:50.989501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746cd38a-a428-46a6-a417-5cfdcb725603 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:51.055455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746cd38a-a428-46a6-a417-5cfdcb725603 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.056178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252c506-24e8-44be-a301-18f8bc2463dd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.060246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be3803-c8d6-4b29-89ff-1a73899b81fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.060899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3803-c8d6-4b29-89ff-1a73899b81fc admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.061185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3803-c8d6-4b29-89ff-1a73899b81fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.061455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be3803-c8d6-4b29-89ff-1a73899b81fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.061632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dba644-3e38-42c0-b954-755dd6856a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.062179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be3803-c8d6-4b29-89ff-1a73899b81fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.062480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dba644-3e38-42c0-b954-755dd6856a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.062901 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dba644-3e38-42c0-b954-755dd6856a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.063304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dba644-3e38-42c0-b954-755dd6856a5c admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.063807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be3803-c8d6-4b29-89ff-1a73899b81fc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.064027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dba644-3e38-42c0-b954-755dd6856a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.066038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dba644-3e38-42c0-b954-755dd6856a5c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.073671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1106/2236] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:38:51.078432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be3803-c8d6-4b29-89ff-1a73899b81fc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:51.088060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1131/2237] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:38:50 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:38:51.088377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be3803-c8d6-4b29-89ff-1a73899b81fc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.092352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dba644-3e38-42c0-b954-755dd6856a5c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:51.094420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54563671-7f6b-4d9f-9163-a0d0b4e17ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.095521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563671-7f6b-4d9f-9163-a0d0b4e17ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.096069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563671-7f6b-4d9f-9163-a0d0b4e17ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.096521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54563671-7f6b-4d9f-9163-a0d0b4e17ea1 admin admin] PATH_INFO: `/v3/projects/e296389d57ef42be9db9e476747c23ea/users/32a605a91297415ab1782b2d71a9502f/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.098547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54563671-7f6b-4d9f-9163-a0d0b4e17ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e296389d57ef42be9db9e476747c23ea, user_id=32a605a91297415ab1782b2d71a9502f, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.101344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54563671-7f6b-4d9f-9163-a0d0b4e17ea1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.102165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dba644-3e38-42c0-b954-755dd6856a5c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.107504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae0d6e-f86a-4f02-ab4f-dafc0b0aaa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.108180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae0d6e-f86a-4f02-ab4f-dafc0b0aaa9d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.108461 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae0d6e-f86a-4f02-ab4f-dafc0b0aaa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.108728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae0d6e-f86a-4f02-ab4f-dafc0b0aaa9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.109455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae0d6e-f86a-4f02-ab4f-dafc0b0aaa9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.111084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae0d6e-f86a-4f02-ab4f-dafc0b0aaa9d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.112465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1107/2238] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:51 2022] PUT /identity/v3/projects/e296389d57ef42be9db9e476747c23ea/users/32a605a91297415ab1782b2d71a9502f/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:38:51.121703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1132/2239] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:38:51.122658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6651ae-6dee-4edd-a2d1-20c42da0359f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.122979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6651ae-6dee-4edd-a2d1-20c42da0359f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.123258 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6651ae-6dee-4edd-a2d1-20c42da0359f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.125596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae0d6e-f86a-4f02-ab4f-dafc0b0aaa9d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:51.136124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae0d6e-f86a-4f02-ab4f-dafc0b0aaa9d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.149889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5a6651ae-6dee-4edd-a2d1-20c42da0359f None None] MFA Rules not processed for user `32a605a91297415ab1782b2d71a9502f`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:51.151117 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5a6651ae-6dee-4edd-a2d1-20c42da0359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:51.168164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:51.197307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d75ac4-06b1-4b85-ac7b-6f5e76015b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.198022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d75ac4-06b1-4b85-ac7b-6f5e76015b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.198307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d75ac4-06b1-4b85-ac7b-6f5e76015b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.198587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d75ac4-06b1-4b85-ac7b-6f5e76015b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.199547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d75ac4-06b1-4b85-ac7b-6f5e76015b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.201173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d75ac4-06b1-4b85-ac7b-6f5e76015b75 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.211871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1133/2240] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:38:51.216669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d75ac4-06b1-4b85-ac7b-6f5e76015b75 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:51.226878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1108/2241] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:51.227150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d75ac4-06b1-4b85-ac7b-6f5e76015b75 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.232140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa19c99-2756-4841-9355-b192d2cdbf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.233030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa19c99-2756-4841-9355-b192d2cdbf86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.233317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa19c99-2756-4841-9355-b192d2cdbf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.233597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa19c99-2756-4841-9355-b192d2cdbf86 admin admin] PATH_INFO: `/v3/projects/7487e0e5a9b04f6f88897825665de1fe/users/51a08d11d6e34491801d7b6f0226cafa/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.235376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6651ae-6dee-4edd-a2d1-20c42da0359f None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:51.237952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5a6651ae-6dee-4edd-a2d1-20c42da0359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:51.250334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa19c99-2756-4841-9355-b192d2cdbf86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7487e0e5a9b04f6f88897825665de1fe, user_id=51a08d11d6e34491801d7b6f0226cafa, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.253074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa19c99-2756-4841-9355-b192d2cdbf86 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.267568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1134/2242] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:51 2022] PUT /identity/v3/projects/7487e0e5a9b04f6f88897825665de1fe/users/51a08d11d6e34491801d7b6f0226cafa/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:51.278638 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797bce1-a9ee-49db-89ea-389251dafc48 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.278967 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797bce1-a9ee-49db-89ea-389251dafc48 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.279315 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797bce1-a9ee-49db-89ea-389251dafc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.299400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6651ae-6dee-4edd-a2d1-20c42da0359f None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.308212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2797bce1-a9ee-49db-89ea-389251dafc48 None None] MFA Rules not processed for user `51a08d11d6e34491801d7b6f0226cafa`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:51.309505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2797bce1-a9ee-49db-89ea-389251dafc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:51.348389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac61d0-9621-4c4f-a1f3-d339cba8b58f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.349050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac61d0-9621-4c4f-a1f3-d339cba8b58f service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.349337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac61d0-9621-4c4f-a1f3-d339cba8b58f service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.349617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac61d0-9621-4c4f-a1f3-d339cba8b58f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.351024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:51.380817 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1135/2243] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:51.392394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797bce1-a9ee-49db-89ea-389251dafc48 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:51.393881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2797bce1-a9ee-49db-89ea-389251dafc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:51.405717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797bce1-a9ee-49db-89ea-389251dafc48 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.409365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b0931-7fcf-4a75-abe7-6220a70d6612 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.409995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b0931-7fcf-4a75-abe7-6220a70d6612 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.410278 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b0931-7fcf-4a75-abe7-6220a70d6612 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.410551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b0931-7fcf-4a75-abe7-6220a70d6612 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.413148 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac61d0-9621-4c4f-a1f3-d339cba8b58f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.414825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac61d0-9621-4c4f-a1f3-d339cba8b58f service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.419106 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b0931-7fcf-4a75-abe7-6220a70d6612 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.420475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b0931-7fcf-4a75-abe7-6220a70d6612 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.424677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1109/2244] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:38:51.431079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1136/2245] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:38:51.508680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac61d0-9621-4c4f-a1f3-d339cba8b58f service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:51.530875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:51.587771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac61d0-9621-4c4f-a1f3-d339cba8b58f service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.597979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b0931-7fcf-4a75-abe7-6220a70d6612 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:51.599598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-482b0931-7fcf-4a75-abe7-6220a70d6612 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:51.600600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5cbc7-46f6-4de4-8596-ff6de7c380fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.601910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5cbc7-46f6-4de4-8596-ff6de7c380fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.604456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5cbc7-46f6-4de4-8596-ff6de7c380fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.605020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5cbc7-46f6-4de4-8596-ff6de7c380fd admin admin] PATH_INFO: `/v3/users/c61d9bf0b11449d4a60859170d9d570f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.606767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5cbc7-46f6-4de4-8596-ff6de7c380fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c61d9bf0b11449d4a60859170d9d570f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.610438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5cbc7-46f6-4de4-8596-ff6de7c380fd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.615968 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b0931-7fcf-4a75-abe7-6220a70d6612 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.624374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66219a7c-3a20-49d8-a5f1-b5f5895b3499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.625849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66219a7c-3a20-49d8-a5f1-b5f5895b3499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.626266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66219a7c-3a20-49d8-a5f1-b5f5895b3499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.626517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66219a7c-3a20-49d8-a5f1-b5f5895b3499 admin admin] PATH_INFO: `/v3/users/adcd31f61c374c58bc56a6e74f13a9d4` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.628238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66219a7c-3a20-49d8-a5f1-b5f5895b3499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adcd31f61c374c58bc56a6e74f13a9d4)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.631100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66219a7c-3a20-49d8-a5f1-b5f5895b3499 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.678511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8cd5cbc7-46f6-4de4-8596-ff6de7c380fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c61d9bf0b11449d4a60859170d9d570f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:51.679511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8cd5cbc7-46f6-4de4-8596-ff6de7c380fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c61d9bf0b11449d4a60859170d9d570f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:51.693871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-66219a7c-3a20-49d8-a5f1-b5f5895b3499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adcd31f61c374c58bc56a6e74f13a9d4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:51.695017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-66219a7c-3a20-49d8-a5f1-b5f5895b3499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adcd31f61c374c58bc56a6e74f13a9d4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:51.696152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8cd5cbc7-46f6-4de4-8596-ff6de7c380fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c61d9bf0b11449d4a60859170d9d570f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:51.696850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8cd5cbc7-46f6-4de4-8596-ff6de7c380fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c61d9bf0b11449d4a60859170d9d570f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:51.707916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8cd5cbc7-46f6-4de4-8596-ff6de7c380fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c61d9bf0b11449d4a60859170d9d570f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:51.710522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-66219a7c-3a20-49d8-a5f1-b5f5895b3499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adcd31f61c374c58bc56a6e74f13a9d4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:51.711600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-66219a7c-3a20-49d8-a5f1-b5f5895b3499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adcd31f61c374c58bc56a6e74f13a9d4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:51.726512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1110/2246] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:51 2022] DELETE /identity/v3/users/c61d9bf0b11449d4a60859170d9d570f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:51.729932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-66219a7c-3a20-49d8-a5f1-b5f5895b3499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adcd31f61c374c58bc56a6e74f13a9d4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:51.732270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4028-554a-421e-b170-61b41a1161be None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.732566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4028-554a-421e-b170-61b41a1161be None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.732854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec4028-554a-421e-b170-61b41a1161be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.751919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1137/2247] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:51 2022] DELETE /identity/v3/users/adcd31f61c374c58bc56a6e74f13a9d4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:51.756133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca39261-23a7-4a8d-9e4e-2aaa478b6cf6 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.756595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca39261-23a7-4a8d-9e4e-2aaa478b6cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.756799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca39261-23a7-4a8d-9e4e-2aaa478b6cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.768181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7dec4028-554a-421e-b170-61b41a1161be None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:51.768181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7dec4028-554a-421e-b170-61b41a1161be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:51.797352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2ca39261-23a7-4a8d-9e4e-2aaa478b6cf6 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:51.798776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2ca39261-23a7-4a8d-9e4e-2aaa478b6cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:51.861649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1111/2248] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:51.869872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec4028-554a-421e-b170-61b41a1161be None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:51.871319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7dec4028-554a-421e-b170-61b41a1161be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:51.890994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1138/2249] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:51.903825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca39261-23a7-4a8d-9e4e-2aaa478b6cf6 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:51.905390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2ca39261-23a7-4a8d-9e4e-2aaa478b6cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:51.923851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:51.939860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec4028-554a-421e-b170-61b41a1161be None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.944203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ccbc4-cfd0-48fd-b5c9-894e5a064d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.945039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ccbc4-cfd0-48fd-b5c9-894e5a064d6f service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.945385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ccbc4-cfd0-48fd-b5c9-894e5a064d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.945721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ccbc4-cfd0-48fd-b5c9-894e5a064d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.950651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:51.955517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ccbc4-cfd0-48fd-b5c9-894e5a064d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.957039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ccbc4-cfd0-48fd-b5c9-894e5a064d6f service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.967463 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1112/2250] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:51.967743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca39261-23a7-4a8d-9e4e-2aaa478b6cf6 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:51.971303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4829b7-a99f-46c4-bd9f-8d08e9829149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:51.972019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4829b7-a99f-46c4-bd9f-8d08e9829149 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:51.972306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4829b7-a99f-46c4-bd9f-8d08e9829149 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:51.972585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4829b7-a99f-46c4-bd9f-8d08e9829149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:51.982292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4829b7-a99f-46c4-bd9f-8d08e9829149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:51.983782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4829b7-a99f-46c4-bd9f-8d08e9829149 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:51.993975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1139/2251] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:52.241460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ccbc4-cfd0-48fd-b5c9-894e5a064d6f service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:52.243044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4a8ccbc4-cfd0-48fd-b5c9-894e5a064d6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:52.258253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ccbc4-cfd0-48fd-b5c9-894e5a064d6f service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:52.263511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:52.264633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.265024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.265446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] PATH_INFO: `/v3/projects/033eb927966e44668beb535cd0a23b91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.266742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=033eb927966e44668beb535cd0a23b91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:52.269105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:52.278768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4829b7-a99f-46c4-bd9f-8d08e9829149 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:52.280477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9d4829b7-a99f-46c4-bd9f-8d08e9829149 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:52.304522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4829b7-a99f-46c4-bd9f-8d08e9829149 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:52.312096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:52.313074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.313433 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.313770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] PATH_INFO: `/v3/projects/54c1ecc58b824fb7a32b9f093de9a5e7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.315096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] RBAC: Authorizing `identity:delete_project(project_id=54c1ecc58b824fb7a32b9f093de9a5e7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:52.317395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:52.328579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033eb927966e44668beb535cd0a23b91', 'request_id': 'req-39bc87f0-0291-4858-a6a9-7778e0092e3d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.334795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033eb927966e44668beb535cd0a23b91', 'request_id': 'req-39bc87f0-0291-4858-a6a9-7778e0092e3d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.340849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033eb927966e44668beb535cd0a23b91', 'request_id': 'req-39bc87f0-0291-4858-a6a9-7778e0092e3d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.355110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] The token cache is being invalidate because project 033eb927966e44668beb535cd0a23b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:52.355394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.356001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.357619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1113/2252] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:52 2022] DELETE /identity/v3/projects/033eb927966e44668beb535cd0a23b91 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:52.361503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:52.362821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:52.366956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54c1ecc58b824fb7a32b9f093de9a5e7', 'request_id': 'req-0b40cf5f-76cd-4edf-bd47-053a873d506a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.373452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54c1ecc58b824fb7a32b9f093de9a5e7', 'request_id': 'req-0b40cf5f-76cd-4edf-bd47-053a873d506a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.380895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54c1ecc58b824fb7a32b9f093de9a5e7', 'request_id': 'req-0b40cf5f-76cd-4edf-bd47-053a873d506a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.397426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] The token cache is being invalidate because project 54c1ecc58b824fb7a32b9f093de9a5e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:52.398075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.398709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0b40cf5f-76cd-4edf-bd47-053a873d506a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.400292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1140/2253] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:52 2022] DELETE /identity/v3/projects/54c1ecc58b824fb7a32b9f093de9a5e7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:52.405019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5c5c6-4bf2-4cb8-9d78-91cb63e27ab1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.405378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5c5c6-4bf2-4cb8-9d78-91cb63e27ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.405721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5c5c6-4bf2-4cb8-9d78-91cb63e27ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.423945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c9a5c5c6-4bf2-4cb8-9d78-91cb63e27ab1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:52.425136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c9a5c5c6-4bf2-4cb8-9d78-91cb63e27ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:52.427923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc87f0-0291-4858-a6a9-7778e0092e3d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:52.490680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:52.490978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1141/2254] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:52.498804 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.499080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.499357 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] PATH_INFO: `/v3/users/32a605a91297415ab1782b2d71a9502f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.499569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153603-d645-41cd-8535-d0700a9f2cba None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.499858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153603-d645-41cd-8535-d0700a9f2cba None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.500134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c153603-d645-41cd-8535-d0700a9f2cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.501111 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a605a91297415ab1782b2d71a9502f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:52.503197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:52.518228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4c153603-d645-41cd-8535-d0700a9f2cba None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:52.518742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:52.519445 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4c153603-d645-41cd-8535-d0700a9f2cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:52.529621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1142/2255] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:52.534448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153603-d645-41cd-8535-d0700a9f2cba None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:52.544293 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c153603-d645-41cd-8535-d0700a9f2cba None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:52.549485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233df9b-eff4-4d35-bde8-730cbaf82b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:52.550268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233df9b-eff4-4d35-bde8-730cbaf82b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.550597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233df9b-eff4-4d35-bde8-730cbaf82b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.550876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e233df9b-eff4-4d35-bde8-730cbaf82b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.551608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233df9b-eff4-4d35-bde8-730cbaf82b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:38:52.551923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233df9b-eff4-4d35-bde8-730cbaf82b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:52.553529 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e233df9b-eff4-4d35-bde8-730cbaf82b14 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:52.560827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a605a91297415ab1782b2d71a9502f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.561501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a605a91297415ab1782b2d71a9502f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.566314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1143/2256] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:38:52.570710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233df9b-eff4-4d35-bde8-730cbaf82b14 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:52.572359 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e233df9b-eff4-4d35-bde8-730cbaf82b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:52.573840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a605a91297415ab1782b2d71a9502f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.574548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a605a91297415ab1782b2d71a9502f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.584371 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a605a91297415ab1782b2d71a9502f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.585568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e233df9b-eff4-4d35-bde8-730cbaf82b14 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:52.590717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562fd4a-07f6-43b1-8617-b8d47578fb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:52.591377 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562fd4a-07f6-43b1-8617-b8d47578fb17 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.591657 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562fd4a-07f6-43b1-8617-b8d47578fb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.591930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562fd4a-07f6-43b1-8617-b8d47578fb17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.592955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562fd4a-07f6-43b1-8617-b8d47578fb17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:52.594872 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562fd4a-07f6-43b1-8617-b8d47578fb17 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:52.598310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1114/2257] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:52 2022] DELETE /identity/v3/users/32a605a91297415ab1782b2d71a9502f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:52.634753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1144/2258] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:38:52 2022] POST /identity/v3/projects => generated 391 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:38:52.639030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:52.703436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261bb0a4-fee7-45c1-9b7c-52f0dbec4e97 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:52.769713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b12f4-92f6-4394-833a-c5edaec33222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:52.770435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b12f4-92f6-4394-833a-c5edaec33222 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.770730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b12f4-92f6-4394-833a-c5edaec33222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.771054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b12f4-92f6-4394-833a-c5edaec33222 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.771769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b12f4-92f6-4394-833a-c5edaec33222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:52.774054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b12f4-92f6-4394-833a-c5edaec33222 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:52.799962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1115/2259] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:38:52 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:38:52.801182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562fd4a-07f6-43b1-8617-b8d47578fb17 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:52.812989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562fd4a-07f6-43b1-8617-b8d47578fb17 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:52.821518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ff684-2aa1-4455-b0a0-85e4fb104a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:52.822307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ff684-2aa1-4455-b0a0-85e4fb104a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.822543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ff684-2aa1-4455-b0a0-85e4fb104a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.822853 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ff684-2aa1-4455-b0a0-85e4fb104a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.823627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ff684-2aa1-4455-b0a0-85e4fb104a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:52.825683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ff684-2aa1-4455-b0a0-85e4fb104a33 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:52.837273 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1145/2260] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:38:52.841755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b12f4-92f6-4394-833a-c5edaec33222 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:52.853265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b12f4-92f6-4394-833a-c5edaec33222 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:52.859118 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e15964-3201-47c6-93d3-e1f31b3f1f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:52.859980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e15964-3201-47c6-93d3-e1f31b3f1f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.860434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e15964-3201-47c6-93d3-e1f31b3f1f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.860996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e15964-3201-47c6-93d3-e1f31b3f1f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.869751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e15964-3201-47c6-93d3-e1f31b3f1f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:52.869751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e15964-3201-47c6-93d3-e1f31b3f1f4f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:52.875594 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1116/2261] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:38:52.879574 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ff684-2aa1-4455-b0a0-85e4fb104a33 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:52.885811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e15964-3201-47c6-93d3-e1f31b3f1f4f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:52.887686 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-12e15964-3201-47c6-93d3-e1f31b3f1f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:52.889884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ff684-2aa1-4455-b0a0-85e4fb104a33 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:52.897046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c208670-e5ff-4e73-a264-cee0ac5bd39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:52.898825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c208670-e5ff-4e73-a264-cee0ac5bd39c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.899477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c208670-e5ff-4e73-a264-cee0ac5bd39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.899760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c208670-e5ff-4e73-a264-cee0ac5bd39c admin admin] PATH_INFO: `/v3/projects/4fc380f695504800a6debba4c78613be/users/1b30cee49c0542dbbb6775b37e4d60bb/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.902064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e15964-3201-47c6-93d3-e1f31b3f1f4f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:52.909117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:52.909998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:52.911876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.912162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.912450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] PATH_INFO: `/v3/projects/e296389d57ef42be9db9e476747c23ea` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.914120 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e296389d57ef42be9db9e476747c23ea)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:52.916799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:52.919230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c208670-e5ff-4e73-a264-cee0ac5bd39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc380f695504800a6debba4c78613be, user_id=1b30cee49c0542dbbb6775b37e4d60bb, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:52.922618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c208670-e5ff-4e73-a264-cee0ac5bd39c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:52.933130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:52.941323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1146/2262] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:52 2022] PUT /identity/v3/projects/4fc380f695504800a6debba4c78613be/users/1b30cee49c0542dbbb6775b37e4d60bb/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:52.953610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b06f9-b100-47d5-b1d3-fc526f72ab77 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:52.954009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b06f9-b100-47d5-b1d3-fc526f72ab77 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:52.954373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b06f9-b100-47d5-b1d3-fc526f72ab77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:52.981774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e296389d57ef42be9db9e476747c23ea', 'request_id': 'req-a460ec5b-c140-4203-8527-52d1733ec1f0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.985613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-670b06f9-b100-47d5-b1d3-fc526f72ab77 None None] MFA Rules not processed for user `1b30cee49c0542dbbb6775b37e4d60bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:52.988769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e296389d57ef42be9db9e476747c23ea', 'request_id': 'req-a460ec5b-c140-4203-8527-52d1733ec1f0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:52.989649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-670b06f9-b100-47d5-b1d3-fc526f72ab77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:52.995195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e296389d57ef42be9db9e476747c23ea', 'request_id': 'req-a460ec5b-c140-4203-8527-52d1733ec1f0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:53.011642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] The token cache is being invalidate because project e296389d57ef42be9db9e476747c23ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:53.012040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:53.012698 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a460ec5b-c140-4203-8527-52d1733ec1f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:53.014811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1117/2263] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:52 2022] DELETE /identity/v3/projects/e296389d57ef42be9db9e476747c23ea => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:38:53.019809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59732dc5-b7ee-4fc9-bc04-c8423020458a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.020243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59732dc5-b7ee-4fc9-bc04-c8423020458a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.020528 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59732dc5-b7ee-4fc9-bc04-c8423020458a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.041097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-59732dc5-b7ee-4fc9-bc04-c8423020458a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:53.042324 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-59732dc5-b7ee-4fc9-bc04-c8423020458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:53.062955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1147/2264] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:53.075678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b06f9-b100-47d5-b1d3-fc526f72ab77 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.079079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-670b06f9-b100-47d5-b1d3-fc526f72ab77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:53.116770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1118/2265] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:53.125594 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77ec20-8409-42cb-8854-2349a2225928 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.125882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77ec20-8409-42cb-8854-2349a2225928 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.126160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77ec20-8409-42cb-8854-2349a2225928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.144285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fe77ec20-8409-42cb-8854-2349a2225928 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:53.145492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fe77ec20-8409-42cb-8854-2349a2225928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:53.153924 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1119/2266] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:53.155122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b06f9-b100-47d5-b1d3-fc526f72ab77 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:53.158128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77ec20-8409-42cb-8854-2349a2225928 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.159396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b041046-cc20-4d6b-87e5-f49d3e98a6e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:53.160070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b041046-cc20-4d6b-87e5-f49d3e98a6e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.160383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b041046-cc20-4d6b-87e5-f49d3e98a6e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.160660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b041046-cc20-4d6b-87e5-f49d3e98a6e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.168116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77ec20-8409-42cb-8854-2349a2225928 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:53.171435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b041046-cc20-4d6b-87e5-f49d3e98a6e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:53.173184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159df80f-d48b-48d4-a4a2-903d99c3df30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:53.173375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b041046-cc20-4d6b-87e5-f49d3e98a6e9 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:53.173977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159df80f-d48b-48d4-a4a2-903d99c3df30 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.174262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159df80f-d48b-48d4-a4a2-903d99c3df30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.174538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159df80f-d48b-48d4-a4a2-903d99c3df30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.175313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159df80f-d48b-48d4-a4a2-903d99c3df30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:38:53.175621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159df80f-d48b-48d4-a4a2-903d99c3df30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:53.177214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159df80f-d48b-48d4-a4a2-903d99c3df30 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:53.184185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1148/2267] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:38:53.189923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1120/2268] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:38:53.197993 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b041046-cc20-4d6b-87e5-f49d3e98a6e9 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.208790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b041046-cc20-4d6b-87e5-f49d3e98a6e9 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:53.214313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975dd33-5003-43d5-a710-6bdf81a03031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:53.215336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975dd33-5003-43d5-a710-6bdf81a03031 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.215618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975dd33-5003-43d5-a710-6bdf81a03031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.215891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975dd33-5003-43d5-a710-6bdf81a03031 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.216593 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975dd33-5003-43d5-a710-6bdf81a03031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:53.218609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975dd33-5003-43d5-a710-6bdf81a03031 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:53.250733 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1149/2269] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:38:53 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:38:53.252208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159df80f-d48b-48d4-a4a2-903d99c3df30 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.263707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159df80f-d48b-48d4-a4a2-903d99c3df30 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:53.324274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19bb3b-ba5f-44eb-809c-b4423dd39222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:53.324993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19bb3b-ba5f-44eb-809c-b4423dd39222 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.325269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19bb3b-ba5f-44eb-809c-b4423dd39222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.325638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19bb3b-ba5f-44eb-809c-b4423dd39222 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.326252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19bb3b-ba5f-44eb-809c-b4423dd39222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:53.328232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19bb3b-ba5f-44eb-809c-b4423dd39222 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:53.353651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1121/2270] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:38:53 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:38:53.358387 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975dd33-5003-43d5-a710-6bdf81a03031 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.369620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975dd33-5003-43d5-a710-6bdf81a03031 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:53.374370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a530a2-184d-49fc-b5d9-ba7b708d00f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:53.375195 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a530a2-184d-49fc-b5d9-ba7b708d00f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.375482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a530a2-184d-49fc-b5d9-ba7b708d00f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.375769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a530a2-184d-49fc-b5d9-ba7b708d00f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.376606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a530a2-184d-49fc-b5d9-ba7b708d00f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:53.378325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a530a2-184d-49fc-b5d9-ba7b708d00f4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:53.392755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1150/2271] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:38:53.397173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19bb3b-ba5f-44eb-809c-b4423dd39222 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.411165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19bb3b-ba5f-44eb-809c-b4423dd39222 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:53.416891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1d31b-7061-46a9-954e-988f61a798fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:53.417641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1d31b-7061-46a9-954e-988f61a798fd admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.417993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1d31b-7061-46a9-954e-988f61a798fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.418400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1d31b-7061-46a9-954e-988f61a798fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.419255 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1d31b-7061-46a9-954e-988f61a798fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:53.420874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1d31b-7061-46a9-954e-988f61a798fd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:53.426736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:53.434737 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1122/2272] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:38:53.438801 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a530a2-184d-49fc-b5d9-ba7b708d00f4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.451129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a530a2-184d-49fc-b5d9-ba7b708d00f4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:53.459784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258461f-bdac-4faa-8002-479ff57aec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:53.461797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258461f-bdac-4faa-8002-479ff57aec1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.462709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258461f-bdac-4faa-8002-479ff57aec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.464746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258461f-bdac-4faa-8002-479ff57aec1d admin admin] PATH_INFO: `/v3/projects/a539ab1ba98b4b93956a94a31c8e76ee/users/123d7d85355847cfbba261b54b6ee7d9/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.509388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258461f-bdac-4faa-8002-479ff57aec1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a539ab1ba98b4b93956a94a31c8e76ee, user_id=123d7d85355847cfbba261b54b6ee7d9, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:53.513150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b258461f-bdac-4faa-8002-479ff57aec1d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:53.536651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1151/2273] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:53 2022] PUT /identity/v3/projects/a539ab1ba98b4b93956a94a31c8e76ee/users/123d7d85355847cfbba261b54b6ee7d9/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:38:53.547679 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b85e5-3b0e-434d-a179-f24fab0602f2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.548490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b85e5-3b0e-434d-a179-f24fab0602f2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.548954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b85e5-3b0e-434d-a179-f24fab0602f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.582153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2c6b85e5-3b0e-434d-a179-f24fab0602f2 None None] MFA Rules not processed for user `123d7d85355847cfbba261b54b6ee7d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:53.583697 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2c6b85e5-3b0e-434d-a179-f24fab0602f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:53.655251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1123/2274] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:53.656453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258461f-bdac-4faa-8002-479ff57aec1d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.686648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:53.734079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258461f-bdac-4faa-8002-479ff57aec1d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:53.739096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a4eca-6272-42be-a763-5d1352e6790c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:53.739870 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a4eca-6272-42be-a763-5d1352e6790c admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.740148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a4eca-6272-42be-a763-5d1352e6790c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.740423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a4eca-6272-42be-a763-5d1352e6790c admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.741152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a4eca-6272-42be-a763-5d1352e6790c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:53.743469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a4eca-6272-42be-a763-5d1352e6790c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:53.775327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1152/2275] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:38:53 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:38:53.776451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b85e5-3b0e-434d-a179-f24fab0602f2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.787920 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b85e5-3b0e-434d-a179-f24fab0602f2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:53.845615 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25b1bd-5970-4a9a-90df-3851bb31c845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:53.846298 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25b1bd-5970-4a9a-90df-3851bb31c845 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.846588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25b1bd-5970-4a9a-90df-3851bb31c845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.846903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25b1bd-5970-4a9a-90df-3851bb31c845 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.847613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25b1bd-5970-4a9a-90df-3851bb31c845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:53.849554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25b1bd-5970-4a9a-90df-3851bb31c845 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:53.878039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1124/2276] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:38:53 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:38:53.879880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a4eca-6272-42be-a763-5d1352e6790c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.890391 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a4eca-6272-42be-a763-5d1352e6790c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:53.896439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c60ed-497a-428b-8bf4-e2f3553d99e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:53.897108 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c60ed-497a-428b-8bf4-e2f3553d99e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.897384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c60ed-497a-428b-8bf4-e2f3553d99e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.897773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c60ed-497a-428b-8bf4-e2f3553d99e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.898718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c60ed-497a-428b-8bf4-e2f3553d99e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:53.900329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c60ed-497a-428b-8bf4-e2f3553d99e6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:53.911887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1153/2277] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:38:53.920900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25b1bd-5970-4a9a-90df-3851bb31c845 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.931957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25b1bd-5970-4a9a-90df-3851bb31c845 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:53.936793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cf1ac-bb66-496b-889e-7365e78164e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:53.937708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cf1ac-bb66-496b-889e-7365e78164e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:53.937994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cf1ac-bb66-496b-889e-7365e78164e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:53.938271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cf1ac-bb66-496b-889e-7365e78164e4 admin admin] PATH_INFO: `/v3/projects/66bb45ae46884a2aa76bc67a97e7a1a8/users/3faa8529899f41758cd47258defea549/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:53.953441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cf1ac-bb66-496b-889e-7365e78164e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bb45ae46884a2aa76bc67a97e7a1a8, user_id=3faa8529899f41758cd47258defea549, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:53.956131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cf1ac-bb66-496b-889e-7365e78164e4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:53.971046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1125/2278] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:53 2022] PUT /identity/v3/projects/66bb45ae46884a2aa76bc67a97e7a1a8/users/3faa8529899f41758cd47258defea549/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:53.975543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c60ed-497a-428b-8bf4-e2f3553d99e6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:53.986132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c60ed-497a-428b-8bf4-e2f3553d99e6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:54.048483 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce176239-e7be-46d1-9573-a97e98916ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:54.049159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce176239-e7be-46d1-9573-a97e98916ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:54.049521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce176239-e7be-46d1-9573-a97e98916ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:54.049716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce176239-e7be-46d1-9573-a97e98916ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:54.050850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce176239-e7be-46d1-9573-a97e98916ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:54.052476 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce176239-e7be-46d1-9573-a97e98916ce8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:54.064299 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1154/2279] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:38:54.068681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cf1ac-bb66-496b-889e-7365e78164e4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:54.080087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cf1ac-bb66-496b-889e-7365e78164e4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:54.086176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8819988-d468-438d-96bf-edab47132d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:54.087113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8819988-d468-438d-96bf-edab47132d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:54.087396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8819988-d468-438d-96bf-edab47132d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:54.087674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8819988-d468-438d-96bf-edab47132d7d admin admin] PATH_INFO: `/v3/projects/66bb45ae46884a2aa76bc67a97e7a1a8/users/3faa8529899f41758cd47258defea549/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:54.090244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8819988-d468-438d-96bf-edab47132d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bb45ae46884a2aa76bc67a97e7a1a8, user_id=3faa8529899f41758cd47258defea549, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:54.092998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8819988-d468-438d-96bf-edab47132d7d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:54.106172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1126/2280] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:54 2022] PUT /identity/v3/projects/66bb45ae46884a2aa76bc67a97e7a1a8/users/3faa8529899f41758cd47258defea549/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:54.118047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915b586-57d1-4875-a2a2-e4a3d7bae7d4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:54.118396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915b586-57d1-4875-a2a2-e4a3d7bae7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:54.118768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915b586-57d1-4875-a2a2-e4a3d7bae7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:54.145131 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3915b586-57d1-4875-a2a2-e4a3d7bae7d4 None None] MFA Rules not processed for user `3faa8529899f41758cd47258defea549`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:54.146362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3915b586-57d1-4875-a2a2-e4a3d7bae7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:54.230257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1155/2281] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:54.241673 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8819988-d468-438d-96bf-edab47132d7d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:54.286983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:54.298962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8819988-d468-438d-96bf-edab47132d7d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:54.302476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab63d3-9e5f-462a-9152-cdd19da6775f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:54.303195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab63d3-9e5f-462a-9152-cdd19da6775f service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:54.303481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab63d3-9e5f-462a-9152-cdd19da6775f service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:54.303791 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab63d3-9e5f-462a-9152-cdd19da6775f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:54.313046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab63d3-9e5f-462a-9152-cdd19da6775f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:54.314522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab63d3-9e5f-462a-9152-cdd19da6775f service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:54.324597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1127/2282] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:54.700904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915b586-57d1-4875-a2a2-e4a3d7bae7d4 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:54.721234 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:54.765811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915b586-57d1-4875-a2a2-e4a3d7bae7d4 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:54.770624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:54.771548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:54.771830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:54.772101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] PATH_INFO: `/v3/users/123d7d85355847cfbba261b54b6ee7d9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:54.773454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=123d7d85355847cfbba261b54b6ee7d9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:54.775624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:54.827539 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123d7d85355847cfbba261b54b6ee7d9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:54.828143 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123d7d85355847cfbba261b54b6ee7d9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:54.840516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123d7d85355847cfbba261b54b6ee7d9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:54.841238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123d7d85355847cfbba261b54b6ee7d9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:54.850311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123d7d85355847cfbba261b54b6ee7d9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:54.863197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1156/2283] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:54 2022] DELETE /identity/v3/users/123d7d85355847cfbba261b54b6ee7d9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:54.868048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff904-a6b9-4cfa-9c5a-ed462a56f6c4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:54.868399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff904-a6b9-4cfa-9c5a-ed462a56f6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:54.868773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff904-a6b9-4cfa-9c5a-ed462a56f6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:54.886786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1cfff904-a6b9-4cfa-9c5a-ed462a56f6c4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:54.888151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1cfff904-a6b9-4cfa-9c5a-ed462a56f6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:54.948273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1128/2284] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:54.958195 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:54.959706 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:55.019610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7c064-8426-413b-a800-8d504301e617 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:55.023586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076828e3-b097-4051-9e05-d739c47e6d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:55.024190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076828e3-b097-4051-9e05-d739c47e6d6f service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:55.024476 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076828e3-b097-4051-9e05-d739c47e6d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:55.024751 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076828e3-b097-4051-9e05-d739c47e6d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:55.034827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076828e3-b097-4051-9e05-d739c47e6d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:55.036667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076828e3-b097-4051-9e05-d739c47e6d6f service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:55.046796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1157/2285] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:55.130021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfff904-a6b9-4cfa-9c5a-ed462a56f6c4 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:55.133340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1cfff904-a6b9-4cfa-9c5a-ed462a56f6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:55.148084 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfff904-a6b9-4cfa-9c5a-ed462a56f6c4 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:55.153062 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:55.154049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:55.154397 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:55.154789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] PATH_INFO: `/v3/users/51a08d11d6e34491801d7b6f0226cafa` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:55.156123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a08d11d6e34491801d7b6f0226cafa)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:55.158190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:55.223313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a08d11d6e34491801d7b6f0226cafa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.224095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a08d11d6e34491801d7b6f0226cafa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.238316 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a08d11d6e34491801d7b6f0226cafa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.239113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a08d11d6e34491801d7b6f0226cafa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.248268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a08d11d6e34491801d7b6f0226cafa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.261439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1129/2286] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:55 2022] DELETE /identity/v3/users/51a08d11d6e34491801d7b6f0226cafa => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:55.266570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5412d-dcf4-43dd-8d01-c589d9fc7418 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:55.267017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5412d-dcf4-43dd-8d01-c589d9fc7418 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:55.267415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5412d-dcf4-43dd-8d01-c589d9fc7418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:55.287994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4dd5412d-dcf4-43dd-8d01-c589d9fc7418 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:55.289477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4dd5412d-dcf4-43dd-8d01-c589d9fc7418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:55.325821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:55.358105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:55.359605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:55.376756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1158/2287] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:55.391805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5412d-dcf4-43dd-8d01-c589d9fc7418 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:55.393232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4dd5412d-dcf4-43dd-8d01-c589d9fc7418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:55.455884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:55.457383 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5412d-dcf4-43dd-8d01-c589d9fc7418 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:55.460908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3a8ae-5fc7-4a25-898d-a81c8d485fe4 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:55.461292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7858d-0a69-447b-b001-f9dd7d42d513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:55.461957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7858d-0a69-447b-b001-f9dd7d42d513 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:55.462208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7858d-0a69-447b-b001-f9dd7d42d513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:55.462478 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7858d-0a69-447b-b001-f9dd7d42d513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:55.470485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:55.471585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:55.472221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:55.472653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] PATH_INFO: `/v3/projects/a539ab1ba98b4b93956a94a31c8e76ee` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:55.472763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7858d-0a69-447b-b001-f9dd7d42d513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:55.474175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a539ab1ba98b4b93956a94a31c8e76ee)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:55.474394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7858d-0a69-447b-b001-f9dd7d42d513 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:55.476442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:55.484832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1159/2288] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:55.526323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a539ab1ba98b4b93956a94a31c8e76ee', 'request_id': 'req-83eb6c3e-658d-4617-b400-e82eb44714d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.532720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a539ab1ba98b4b93956a94a31c8e76ee', 'request_id': 'req-83eb6c3e-658d-4617-b400-e82eb44714d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.539301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a539ab1ba98b4b93956a94a31c8e76ee', 'request_id': 'req-83eb6c3e-658d-4617-b400-e82eb44714d8'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.555611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] The token cache is being invalidate because project a539ab1ba98b4b93956a94a31c8e76ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:55.555903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.556576 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.558307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1130/2289] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:55 2022] DELETE /identity/v3/projects/a539ab1ba98b4b93956a94a31c8e76ee => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:38:55.562618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7858d-0a69-447b-b001-f9dd7d42d513 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:55.576078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7858d-0a69-447b-b001-f9dd7d42d513 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:55.653709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49375cb-b7a4-4a5a-8651-d998f86555a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:55.654624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49375cb-b7a4-4a5a-8651-d998f86555a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:55.654934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49375cb-b7a4-4a5a-8651-d998f86555a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:55.655210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49375cb-b7a4-4a5a-8651-d998f86555a4 admin admin] PATH_INFO: `/v3/users/3faa8529899f41758cd47258defea549` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:55.656421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49375cb-b7a4-4a5a-8651-d998f86555a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3faa8529899f41758cd47258defea549)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:55.658351 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49375cb-b7a4-4a5a-8651-d998f86555a4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:55.713915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d49375cb-b7a4-4a5a-8651-d998f86555a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3faa8529899f41758cd47258defea549'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.715247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d49375cb-b7a4-4a5a-8651-d998f86555a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3faa8529899f41758cd47258defea549'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.728743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d49375cb-b7a4-4a5a-8651-d998f86555a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3faa8529899f41758cd47258defea549'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.729648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d49375cb-b7a4-4a5a-8651-d998f86555a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3faa8529899f41758cd47258defea549'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.739038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d49375cb-b7a4-4a5a-8651-d998f86555a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3faa8529899f41758cd47258defea549'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.754491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1160/2290] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:55 2022] DELETE /identity/v3/users/3faa8529899f41758cd47258defea549 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:55.791852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:55.793408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:55.811480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb6c3e-658d-4617-b400-e82eb44714d8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:55.894141 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:55.895116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:55.895396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:55.895670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] PATH_INFO: `/v3/projects/7487e0e5a9b04f6f88897825665de1fe` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:55.897185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7487e0e5a9b04f6f88897825665de1fe)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:55.898884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:55.951874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7487e0e5a9b04f6f88897825665de1fe', 'request_id': 'req-78ae287e-3dba-48da-a088-a9d870da784d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.958710 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7487e0e5a9b04f6f88897825665de1fe', 'request_id': 'req-78ae287e-3dba-48da-a088-a9d870da784d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.965562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7487e0e5a9b04f6f88897825665de1fe', 'request_id': 'req-78ae287e-3dba-48da-a088-a9d870da784d'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.982512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] The token cache is being invalidate because project 7487e0e5a9b04f6f88897825665de1fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:55.982832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.983561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:55.985161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1131/2291] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:55 2022] DELETE /identity/v3/projects/7487e0e5a9b04f6f88897825665de1fe => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:55.990522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6f08-898e-4223-ba9a-e5b97e33bf8a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:55.990838 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6f08-898e-4223-ba9a-e5b97e33bf8a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:55.991121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c6f08-898e-4223-ba9a-e5b97e33bf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:55.997652 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:56.010825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3c7c6f08-898e-4223-ba9a-e5b97e33bf8a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:56.012919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3c7c6f08-898e-4223-ba9a-e5b97e33bf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:56.062289 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.064552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:56.086281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1161/2292] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:56.095676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1433a3d-94bd-4a5e-ab7c-16d3db0ba8ce None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.096227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1433a3d-94bd-4a5e-ab7c-16d3db0ba8ce None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.096521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1433a3d-94bd-4a5e-ab7c-16d3db0ba8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.102289 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:56.125250 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c1433a3d-94bd-4a5e-ab7c-16d3db0ba8ce None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:56.126410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c1433a3d-94bd-4a5e-ab7c-16d3db0ba8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:56.136836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1162/2293] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:56.140996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1433a3d-94bd-4a5e-ab7c-16d3db0ba8ce None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.150522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1433a3d-94bd-4a5e-ab7c-16d3db0ba8ce None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.155316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af1b17-fca3-4726-a9b9-5ab74cf12bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.155943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af1b17-fca3-4726-a9b9-5ab74cf12bbc admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.156059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae287e-3dba-48da-a088-a9d870da784d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.156569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af1b17-fca3-4726-a9b9-5ab74cf12bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.156569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af1b17-fca3-4726-a9b9-5ab74cf12bbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.157526 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af1b17-fca3-4726-a9b9-5ab74cf12bbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:38:56.157887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af1b17-fca3-4726-a9b9-5ab74cf12bbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.159546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af1b17-fca3-4726-a9b9-5ab74cf12bbc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.161631 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.162555 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.163073 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.163647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] PATH_INFO: `/v3/projects/66bb45ae46884a2aa76bc67a97e7a1a8` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.164996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bb45ae46884a2aa76bc67a97e7a1a8)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.167031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.172016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1163/2294] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:38:56.176658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af1b17-fca3-4726-a9b9-5ab74cf12bbc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.186361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af1b17-fca3-4726-a9b9-5ab74cf12bbc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.191057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a98589-5d53-425d-a9fb-2cbf33646490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.191699 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a98589-5d53-425d-a9fb-2cbf33646490 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.191980 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a98589-5d53-425d-a9fb-2cbf33646490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.192258 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a98589-5d53-425d-a9fb-2cbf33646490 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.192937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a98589-5d53-425d-a9fb-2cbf33646490 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.194946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a98589-5d53-425d-a9fb-2cbf33646490 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.219450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bb45ae46884a2aa76bc67a97e7a1a8', 'request_id': 'req-eceef70c-26c0-4c5e-99a8-31fd75822634'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:56.226395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1164/2295] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:38:56 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:38:56.226684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bb45ae46884a2aa76bc67a97e7a1a8', 'request_id': 'req-eceef70c-26c0-4c5e-99a8-31fd75822634'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:56.230157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a98589-5d53-425d-a9fb-2cbf33646490 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.232417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bb45ae46884a2aa76bc67a97e7a1a8', 'request_id': 'req-eceef70c-26c0-4c5e-99a8-31fd75822634'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:56.245940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] The token cache is being invalidate because project 66bb45ae46884a2aa76bc67a97e7a1a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:56.246226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:56.246885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-eceef70c-26c0-4c5e-99a8-31fd75822634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:56.248703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1132/2296] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:56 2022] DELETE /identity/v3/projects/66bb45ae46884a2aa76bc67a97e7a1a8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:38:56.253309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf2316-f149-47bb-b5c5-872afd6ccddf None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.253594 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf2316-f149-47bb-b5c5-872afd6ccddf None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.253871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf2316-f149-47bb-b5c5-872afd6ccddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.272288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d4bf2316-f149-47bb-b5c5-872afd6ccddf None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:56.273600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d4bf2316-f149-47bb-b5c5-872afd6ccddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:56.295297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a98589-5d53-425d-a9fb-2cbf33646490 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.299973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca0c13-58df-4d87-9d53-5046cc745355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.300648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca0c13-58df-4d87-9d53-5046cc745355 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.300930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca0c13-58df-4d87-9d53-5046cc745355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.301202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca0c13-58df-4d87-9d53-5046cc745355 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.301900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca0c13-58df-4d87-9d53-5046cc745355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.303938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca0c13-58df-4d87-9d53-5046cc745355 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.325759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1165/2297] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:38:56 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:38:56.329824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca0c13-58df-4d87-9d53-5046cc745355 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.331446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c2ca0c13-58df-4d87-9d53-5046cc745355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:56.338404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1133/2298] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:56.344039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca0c13-58df-4d87-9d53-5046cc745355 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.346326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4a234-44db-494b-a749-b662f98bd078 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.346617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4a234-44db-494b-a749-b662f98bd078 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.346921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4a234-44db-494b-a749-b662f98bd078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.349775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbabff1-a52a-4984-a08a-68a79caea64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.350446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbabff1-a52a-4984-a08a-68a79caea64f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.351427 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbabff1-a52a-4984-a08a-68a79caea64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.351718 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbabff1-a52a-4984-a08a-68a79caea64f admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.352493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbabff1-a52a-4984-a08a-68a79caea64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.354180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbabff1-a52a-4984-a08a-68a79caea64f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.365181 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1166/2299] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:38:56.365573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-82f4a234-44db-494b-a749-b662f98bd078 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:56.367529 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-82f4a234-44db-494b-a749-b662f98bd078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:56.369594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbabff1-a52a-4984-a08a-68a79caea64f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.376280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1134/2300] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:56.380319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4a234-44db-494b-a749-b662f98bd078 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.382528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbabff1-a52a-4984-a08a-68a79caea64f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.387520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd8497-f379-4c6f-a311-1c4337138631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.388204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd8497-f379-4c6f-a311-1c4337138631 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.388485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd8497-f379-4c6f-a311-1c4337138631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.388757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd8497-f379-4c6f-a311-1c4337138631 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.389464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd8497-f379-4c6f-a311-1c4337138631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.390278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4a234-44db-494b-a749-b662f98bd078 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.391107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd8497-f379-4c6f-a311-1c4337138631 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.394919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc3dc4-87f8-41e7-8a17-c6dfa94c27b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.395553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc3dc4-87f8-41e7-8a17-c6dfa94c27b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.395832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc3dc4-87f8-41e7-8a17-c6dfa94c27b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.396107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc3dc4-87f8-41e7-8a17-c6dfa94c27b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.396837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc3dc4-87f8-41e7-8a17-c6dfa94c27b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:38:56.397139 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc3dc4-87f8-41e7-8a17-c6dfa94c27b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.398875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc3dc4-87f8-41e7-8a17-c6dfa94c27b2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.401877 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1167/2301] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:38:56.406022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd8497-f379-4c6f-a311-1c4337138631 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.410619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1135/2302] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:38:56.414876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc3dc4-87f8-41e7-8a17-c6dfa94c27b2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.416283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd8497-f379-4c6f-a311-1c4337138631 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.421788 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7268df4-d504-42fa-943f-673709da1224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.422683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7268df4-d504-42fa-943f-673709da1224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.423015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7268df4-d504-42fa-943f-673709da1224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.423291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7268df4-d504-42fa-943f-673709da1224 admin admin] PATH_INFO: `/v3/projects/d1cc844419c14acd9a9e7dfb4233841d/users/e3be3323efb945f1ad6537bbc9eba7b0/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.424417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc3dc4-87f8-41e7-8a17-c6dfa94c27b2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.429027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e22a9-f410-4162-b910-d6e12f8eb494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.429758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e22a9-f410-4162-b910-d6e12f8eb494 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.430036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e22a9-f410-4162-b910-d6e12f8eb494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.430309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e22a9-f410-4162-b910-d6e12f8eb494 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.431023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e22a9-f410-4162-b910-d6e12f8eb494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.432909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e22a9-f410-4162-b910-d6e12f8eb494 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.439450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7268df4-d504-42fa-943f-673709da1224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1cc844419c14acd9a9e7dfb4233841d, user_id=e3be3323efb945f1ad6537bbc9eba7b0, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.442114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7268df4-d504-42fa-943f-673709da1224 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.463317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1168/2303] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:56 2022] PUT /identity/v3/projects/d1cc844419c14acd9a9e7dfb4233841d/users/e3be3323efb945f1ad6537bbc9eba7b0/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:38:56.463891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1136/2304] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:38:56 2022] POST /identity/v3/projects => generated 359 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:38:56.465981 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe707c-9324-4f3c-994d-f970115af73f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.466357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe707c-9324-4f3c-994d-f970115af73f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.466672 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe707c-9324-4f3c-994d-f970115af73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.468248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e22a9-f410-4162-b910-d6e12f8eb494 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.475996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:56.477667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:56.491937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-cefe707c-9324-4f3c-994d-f970115af73f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:56.493381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cefe707c-9324-4f3c-994d-f970115af73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:56.539035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e22a9-f410-4162-b910-d6e12f8eb494 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.544476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952e2ac-6859-445d-8742-e14d31aed62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.545177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e2ac-6859-445d-8742-e14d31aed62e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.545445 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e2ac-6859-445d-8742-e14d31aed62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.546185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952e2ac-6859-445d-8742-e14d31aed62e admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.546427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952e2ac-6859-445d-8742-e14d31aed62e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.548409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952e2ac-6859-445d-8742-e14d31aed62e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.570337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1169/2305] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:56.572299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1137/2306] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:38:56 2022] POST /identity/v3/users => generated 280 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:38:56.576228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe707c-9324-4f3c-994d-f970115af73f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.579934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952e2ac-6859-445d-8742-e14d31aed62e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.582026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f952e2ac-6859-445d-8742-e14d31aed62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:56.587689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe707c-9324-4f3c-994d-f970115af73f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.593223 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcc423-beb2-44ef-9871-fbbd3c07bb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.593921 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcc423-beb2-44ef-9871-fbbd3c07bb55 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.594200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcc423-beb2-44ef-9871-fbbd3c07bb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.594462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbcc423-beb2-44ef-9871-fbbd3c07bb55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.595222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcc423-beb2-44ef-9871-fbbd3c07bb55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.597266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbcc423-beb2-44ef-9871-fbbd3c07bb55 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.608854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1170/2307] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:38:56.612828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcc423-beb2-44ef-9871-fbbd3c07bb55 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.624278 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbcc423-beb2-44ef-9871-fbbd3c07bb55 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.629135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb45bd-9d8c-4c5c-935a-d2409a8c9b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.630526 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb45bd-9d8c-4c5c-935a-d2409a8c9b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.631090 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb45bd-9d8c-4c5c-935a-d2409a8c9b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.631206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb45bd-9d8c-4c5c-935a-d2409a8c9b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.632060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb45bd-9d8c-4c5c-935a-d2409a8c9b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.633690 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb45bd-9d8c-4c5c-935a-d2409a8c9b8c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.649299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952e2ac-6859-445d-8742-e14d31aed62e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.653295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d35d9b-f0ce-4ab5-be5f-2e1b24a8ff13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.653949 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1171/2308] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:38:56.654292 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d35d9b-f0ce-4ab5-be5f-2e1b24a8ff13 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.655001 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d35d9b-f0ce-4ab5-be5f-2e1b24a8ff13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.655242 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d35d9b-f0ce-4ab5-be5f-2e1b24a8ff13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.658233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb45bd-9d8c-4c5c-935a-d2409a8c9b8c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.666846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d35d9b-f0ce-4ab5-be5f-2e1b24a8ff13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.668288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d35d9b-f0ce-4ab5-be5f-2e1b24a8ff13 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.670496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb45bd-9d8c-4c5c-935a-d2409a8c9b8c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.675619 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbdee5-a4ac-47ae-998d-9a845786d887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.676754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbdee5-a4ac-47ae-998d-9a845786d887 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.677028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbdee5-a4ac-47ae-998d-9a845786d887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.677298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbdee5-a4ac-47ae-998d-9a845786d887 admin admin] PATH_INFO: `/v3/projects/cd16fbe97ef94115b1f2439884820b8d/users/c80273bedd30428f90c5ecfaf50533fa/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.679243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1138/2309] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:38:56.695162 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbdee5-a4ac-47ae-998d-9a845786d887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd16fbe97ef94115b1f2439884820b8d, user_id=c80273bedd30428f90c5ecfaf50533fa, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.697810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbdee5-a4ac-47ae-998d-9a845786d887 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.716681 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1172/2310] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:56 2022] PUT /identity/v3/projects/cd16fbe97ef94115b1f2439884820b8d/users/c80273bedd30428f90c5ecfaf50533fa/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:38:56.727290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfafb5-4d63-4988-a7b8-500369e6d01b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.727626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfafb5-4d63-4988-a7b8-500369e6d01b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.727909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfafb5-4d63-4988-a7b8-500369e6d01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.755640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c5cfafb5-4d63-4988-a7b8-500369e6d01b None None] MFA Rules not processed for user `c80273bedd30428f90c5ecfaf50533fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:56.756915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c5cfafb5-4d63-4988-a7b8-500369e6d01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:56.842440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1139/2311] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:56.879742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbdee5-a4ac-47ae-998d-9a845786d887 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:56.882243 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-32bbdee5-a4ac-47ae-998d-9a845786d887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:56.956050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbdee5-a4ac-47ae-998d-9a845786d887 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:56.960836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbfc9de-9628-421e-9b0b-d1dd9caed7d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:56.961666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfc9de-9628-421e-9b0b-d1dd9caed7d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:56.961951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfc9de-9628-421e-9b0b-d1dd9caed7d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:56.962308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbfc9de-9628-421e-9b0b-d1dd9caed7d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:56.973252 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfc9de-9628-421e-9b0b-d1dd9caed7d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:56.975076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbfc9de-9628-421e-9b0b-d1dd9caed7d0 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:56.985756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1173/2312] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:56 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:57.001729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfafb5-4d63-4988-a7b8-500369e6d01b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:57.036836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:57.073192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfafb5-4d63-4988-a7b8-500369e6d01b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:57.078073 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:57.079203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:57.079486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:57.079853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] PATH_INFO: `/v3/users/c80273bedd30428f90c5ecfaf50533fa` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:57.081285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c80273bedd30428f90c5ecfaf50533fa)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:57.083381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:57.148804 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c80273bedd30428f90c5ecfaf50533fa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:57.152800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c80273bedd30428f90c5ecfaf50533fa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:57.176639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c80273bedd30428f90c5ecfaf50533fa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:57.177591 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c80273bedd30428f90c5ecfaf50533fa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:57.187082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c80273bedd30428f90c5ecfaf50533fa'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:57.201271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1140/2313] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:57 2022] DELETE /identity/v3/users/c80273bedd30428f90c5ecfaf50533fa => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:57.205587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9c9aa-f552-45ad-8edd-6461d03de7b4 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:57.205873 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9c9aa-f552-45ad-8edd-6461d03de7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:57.206145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9c9aa-f552-45ad-8edd-6461d03de7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:57.226466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-79e9c9aa-f552-45ad-8edd-6461d03de7b4 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:57.227767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-79e9c9aa-f552-45ad-8edd-6461d03de7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:57.294759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1174/2314] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:57.320481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:57.321951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:57.408859 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191e913-ac6b-4eb1-a707-3d7ac5191fe7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:57.414561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e5b2f-ca77-4b39-b197-de95882b3875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:57.415757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e5b2f-ca77-4b39-b197-de95882b3875 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:57.416674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e5b2f-ca77-4b39-b197-de95882b3875 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:57.417346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e5b2f-ca77-4b39-b197-de95882b3875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:57.417633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a9aed-9cde-4a94-93d9-d2c7f2a49849 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:57.418044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a9aed-9cde-4a94-93d9-d2c7f2a49849 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:57.418454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a9aed-9cde-4a94-93d9-d2c7f2a49849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:57.424653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:57.430596 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e5b2f-ca77-4b39-b197-de95882b3875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:57.432818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e5b2f-ca77-4b39-b197-de95882b3875 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:57.447964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1141/2315] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:57.457755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-446a9aed-9cde-4a94-93d9-d2c7f2a49849 None None] MFA Rules not processed for user `e3be3323efb945f1ad6537bbc9eba7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:57.459067 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-446a9aed-9cde-4a94-93d9-d2c7f2a49849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:57.541704 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1175/2316] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:57.553580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e5b2f-ca77-4b39-b197-de95882b3875 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:57.555361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-024e5b2f-ca77-4b39-b197-de95882b3875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:57.624123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e5b2f-ca77-4b39-b197-de95882b3875 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:57.628229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62144318-60ab-4c99-87b6-382ec67ab2df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:57.629406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62144318-60ab-4c99-87b6-382ec67ab2df service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:57.629698 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62144318-60ab-4c99-87b6-382ec67ab2df service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:57.629981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62144318-60ab-4c99-87b6-382ec67ab2df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:57.635729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:57.639661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62144318-60ab-4c99-87b6-382ec67ab2df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:57.642075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62144318-60ab-4c99-87b6-382ec67ab2df service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:57.662761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1142/2317] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:58.354484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a9aed-9cde-4a94-93d9-d2c7f2a49849 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:58.356151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-446a9aed-9cde-4a94-93d9-d2c7f2a49849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:58.370216 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a9aed-9cde-4a94-93d9-d2c7f2a49849 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:58.374884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:58.375705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:58.376000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:58.376266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] PATH_INFO: `/v3/projects/cd16fbe97ef94115b1f2439884820b8d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:58.377879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd16fbe97ef94115b1f2439884820b8d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:58.379681 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:58.433261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd16fbe97ef94115b1f2439884820b8d', 'request_id': 'req-1d2ab87a-247d-4d58-8222-27084a46a7db'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:58.439915 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd16fbe97ef94115b1f2439884820b8d', 'request_id': 'req-1d2ab87a-247d-4d58-8222-27084a46a7db'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:58.447602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd16fbe97ef94115b1f2439884820b8d', 'request_id': 'req-1d2ab87a-247d-4d58-8222-27084a46a7db'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:58.468046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] The token cache is being invalidate because project cd16fbe97ef94115b1f2439884820b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:58.468347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:58.468999 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1d2ab87a-247d-4d58-8222-27084a46a7db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:58.470944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1176/2318] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:58 2022] DELETE /identity/v3/projects/cd16fbe97ef94115b1f2439884820b8d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:58.476195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6508327-dd05-4103-a574-ec9ecf423006 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:58.476718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6508327-dd05-4103-a574-ec9ecf423006 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:58.476718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6508327-dd05-4103-a574-ec9ecf423006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:58.494980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c6508327-dd05-4103-a574-ec9ecf423006 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:58.496181 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c6508327-dd05-4103-a574-ec9ecf423006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:58.563611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1143/2319] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:38:58.572569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672de6f-7fda-408e-92a1-10c055dda45f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:58.572861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672de6f-7fda-408e-92a1-10c055dda45f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:58.573157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672de6f-7fda-408e-92a1-10c055dda45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:58.593172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2672de6f-7fda-408e-92a1-10c055dda45f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:58.594416 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2672de6f-7fda-408e-92a1-10c055dda45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:58.603652 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1177/2320] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:58.608279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6508327-dd05-4103-a574-ec9ecf423006 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:58.618728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6508327-dd05-4103-a574-ec9ecf423006 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:58.624389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d57ec-13b4-46ec-a5b9-d4833739c0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:58.625085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d57ec-13b4-46ec-a5b9-d4833739c0fc admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:58.625367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d57ec-13b4-46ec-a5b9-d4833739c0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:58.625638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d57ec-13b4-46ec-a5b9-d4833739c0fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:58.626371 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d57ec-13b4-46ec-a5b9-d4833739c0fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:38:58.626754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d57ec-13b4-46ec-a5b9-d4833739c0fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:58.628533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d57ec-13b4-46ec-a5b9-d4833739c0fc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:58.642014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1144/2321] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:38:58.646873 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672de6f-7fda-408e-92a1-10c055dda45f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:58.657162 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672de6f-7fda-408e-92a1-10c055dda45f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:58.662072 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7c939-2c19-4036-9d05-e03c35b924a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:58.662820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7c939-2c19-4036-9d05-e03c35b924a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:58.663604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7c939-2c19-4036-9d05-e03c35b924a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:58.663897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7c939-2c19-4036-9d05-e03c35b924a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:58.664606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7c939-2c19-4036-9d05-e03c35b924a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:58.666543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7c939-2c19-4036-9d05-e03c35b924a6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:58.695998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1178/2322] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:38:58 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:38:58.700175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d57ec-13b4-46ec-a5b9-d4833739c0fc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:58.710509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d57ec-13b4-46ec-a5b9-d4833739c0fc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:58.768936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b93fdd-fdbf-4a71-bbc2-6b5f2050217c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:58.769614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b93fdd-fdbf-4a71-bbc2-6b5f2050217c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:58.769893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b93fdd-fdbf-4a71-bbc2-6b5f2050217c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:58.770166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b93fdd-fdbf-4a71-bbc2-6b5f2050217c admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:58.770912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b93fdd-fdbf-4a71-bbc2-6b5f2050217c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:58.772935 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b93fdd-fdbf-4a71-bbc2-6b5f2050217c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:58.795416 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1145/2323] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:38:58 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:38:58.799514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7c939-2c19-4036-9d05-e03c35b924a6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:58.810271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7c939-2c19-4036-9d05-e03c35b924a6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:58.815562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b7bc4-491d-4578-8977-ab4b9b09750e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:58.816231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b7bc4-491d-4578-8977-ab4b9b09750e admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:58.816512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b7bc4-491d-4578-8977-ab4b9b09750e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:58.816787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665b7bc4-491d-4578-8977-ab4b9b09750e admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:58.817504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b7bc4-491d-4578-8977-ab4b9b09750e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:58.819193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665b7bc4-491d-4578-8977-ab4b9b09750e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:58.834623 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1179/2324] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:38:58.838787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b93fdd-fdbf-4a71-bbc2-6b5f2050217c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:58.849489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b93fdd-fdbf-4a71-bbc2-6b5f2050217c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:58.854393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b2155-959f-44bd-a69f-cc64a5764eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:58.855096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b2155-959f-44bd-a69f-cc64a5764eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:58.855381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b2155-959f-44bd-a69f-cc64a5764eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:58.855658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b2155-959f-44bd-a69f-cc64a5764eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:58.856399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b2155-959f-44bd-a69f-cc64a5764eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:58.858013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b2155-959f-44bd-a69f-cc64a5764eb4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:58.868450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1146/2325] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:38:58.874314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b7bc4-491d-4578-8977-ab4b9b09750e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:58.885527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665b7bc4-491d-4578-8977-ab4b9b09750e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:58.891049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08717-1c6b-44f9-ab65-f6a15e7234a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:58.891972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08717-1c6b-44f9-ab65-f6a15e7234a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:58.892246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08717-1c6b-44f9-ab65-f6a15e7234a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:58.892527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e08717-1c6b-44f9-ab65-f6a15e7234a4 admin admin] PATH_INFO: `/v3/projects/6ae82dc127d74728b990b45a02cb673b/users/273a3dc28d46401fbbe6986c1a6d5464/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:58.908433 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e08717-1c6b-44f9-ab65-f6a15e7234a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae82dc127d74728b990b45a02cb673b, user_id=273a3dc28d46401fbbe6986c1a6d5464, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:58.911133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e08717-1c6b-44f9-ab65-f6a15e7234a4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:58.926722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1180/2326] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:38:58 2022] PUT /identity/v3/projects/6ae82dc127d74728b990b45a02cb673b/users/273a3dc28d46401fbbe6986c1a6d5464/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:58.936829 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28ae37-4e7b-4c0a-8fcc-4254df7b1d48 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:58.937218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28ae37-4e7b-4c0a-8fcc-4254df7b1d48 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:58.937554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc28ae37-4e7b-4c0a-8fcc-4254df7b1d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:58.966610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fc28ae37-4e7b-4c0a-8fcc-4254df7b1d48 None None] MFA Rules not processed for user `273a3dc28d46401fbbe6986c1a6d5464`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:58.967884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fc28ae37-4e7b-4c0a-8fcc-4254df7b1d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:59.032836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:59.052452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1147/2327] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:59.063913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08717-1c6b-44f9-ab65-f6a15e7234a4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:59.065612 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-27e08717-1c6b-44f9-ab65-f6a15e7234a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:59.079423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:38:59.126352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e08717-1c6b-44f9-ab65-f6a15e7234a4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:59.130109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d77ac3-1a7c-46d4-afb7-8c29e791de77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:59.131207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d77ac3-1a7c-46d4-afb7-8c29e791de77 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:59.131207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d77ac3-1a7c-46d4-afb7-8c29e791de77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:59.131523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d77ac3-1a7c-46d4-afb7-8c29e791de77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:59.140134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d77ac3-1a7c-46d4-afb7-8c29e791de77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:59.141575 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d77ac3-1a7c-46d4-afb7-8c29e791de77 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:59.150644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1181/2328] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:59.177336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28ae37-4e7b-4c0a-8fcc-4254df7b1d48 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:59.252454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc28ae37-4e7b-4c0a-8fcc-4254df7b1d48 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:59.257287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:59.258191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:59.258472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:59.258779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] PATH_INFO: `/v3/users/273a3dc28d46401fbbe6986c1a6d5464` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:59.260145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=273a3dc28d46401fbbe6986c1a6d5464)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:59.262160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:59.314381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '273a3dc28d46401fbbe6986c1a6d5464'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:59.315048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '273a3dc28d46401fbbe6986c1a6d5464'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:59.327925 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '273a3dc28d46401fbbe6986c1a6d5464'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:59.328667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '273a3dc28d46401fbbe6986c1a6d5464'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:59.337795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '273a3dc28d46401fbbe6986c1a6d5464'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:59.352499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1148/2329] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:38:59 2022] DELETE /identity/v3/users/273a3dc28d46401fbbe6986c1a6d5464 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:38:59.356892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c659333-58a9-4c39-b53c-ca763cd57ac5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:59.357372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c659333-58a9-4c39-b53c-ca763cd57ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:59.357567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c659333-58a9-4c39-b53c-ca763cd57ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:59.376892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0c659333-58a9-4c39-b53c-ca763cd57ac5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:38:59.378240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c659333-58a9-4c39-b53c-ca763cd57ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:59.442187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1182/2330] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:38:59.452329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:59.453757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:59.514156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae978dc-ffb6-49c6-903d-8b57e19f6419 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:59.518760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df8c9c-802b-449e-8674-c38d83216fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:59.519496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df8c9c-802b-449e-8674-c38d83216fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:59.519796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df8c9c-802b-449e-8674-c38d83216fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:59.520059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df8c9c-802b-449e-8674-c38d83216fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:59.530370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df8c9c-802b-449e-8674-c38d83216fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:59.531844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df8c9c-802b-449e-8674-c38d83216fdf service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:59.541916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1149/2331] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:38:59.834875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c659333-58a9-4c39-b53c-ca763cd57ac5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:38:59.836401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c659333-58a9-4c39-b53c-ca763cd57ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:38:59.850716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c659333-58a9-4c39-b53c-ca763cd57ac5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:38:59.856712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:38:59.857865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:38:59.858079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:38:59.858395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] PATH_INFO: `/v3/projects/6ae82dc127d74728b990b45a02cb673b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:38:59.859855 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae82dc127d74728b990b45a02cb673b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:38:59.861734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:38:59.919232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae82dc127d74728b990b45a02cb673b', 'request_id': 'req-8f438409-4856-4a93-8ee7-5c7be446111a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:59.928583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae82dc127d74728b990b45a02cb673b', 'request_id': 'req-8f438409-4856-4a93-8ee7-5c7be446111a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:59.936333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae82dc127d74728b990b45a02cb673b', 'request_id': 'req-8f438409-4856-4a93-8ee7-5c7be446111a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:59.951807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] The token cache is being invalidate because project 6ae82dc127d74728b990b45a02cb673b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:38:59.952060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:59.952764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:38:59.954360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1183/2332] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:38:59 2022] DELETE /identity/v3/projects/6ae82dc127d74728b990b45a02cb673b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:39:02.194639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583564b5-9acb-4722-aa43-f9ce133cb1d7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:02.195183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583564b5-9acb-4722-aa43-f9ce133cb1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:02.195183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583564b5-9acb-4722-aa43-f9ce133cb1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:02.213395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-583564b5-9acb-4722-aa43-f9ce133cb1d7 None None] MFA Rules not processed for user `e3be3323efb945f1ad6537bbc9eba7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:39:02.214588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-583564b5-9acb-4722-aa43-f9ce133cb1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:02.220306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:39:02.274672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1150/2333] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:39:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:39:09.948893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:39:12.267551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=20 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:39:17.918760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:17.920888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:17.996033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f438409-4856-4a93-8ee7-5c7be446111a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:18.001871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:18.003057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:18.003057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:18.003328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] PATH_INFO: `/v3/users/1b30cee49c0542dbbb6775b37e4d60bb` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:18.004649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b30cee49c0542dbbb6775b37e4d60bb)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:18.006588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:18.061146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b30cee49c0542dbbb6775b37e4d60bb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:39:18.061873 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b30cee49c0542dbbb6775b37e4d60bb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:39:18.074423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b30cee49c0542dbbb6775b37e4d60bb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:39:18.075115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b30cee49c0542dbbb6775b37e4d60bb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:39:18.085640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b30cee49c0542dbbb6775b37e4d60bb'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:39:18.099673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1184/2334] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:39:17 2022] DELETE /identity/v3/users/1b30cee49c0542dbbb6775b37e4d60bb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:39:18.104238 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47ba5d-302d-4600-a512-43071219039b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:18.104676 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47ba5d-302d-4600-a512-43071219039b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:18.104830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47ba5d-302d-4600-a512-43071219039b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:18.123374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8b47ba5d-302d-4600-a512-43071219039b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:39:18.124697 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8b47ba5d-302d-4600-a512-43071219039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:18.183762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1151/2335] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:39:18.193504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:18.195365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:18.267494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e20fdd-e757-40cc-a05a-bf53f139cbae admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:18.272100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517eaf1-667f-4996-997d-0eabf6a94d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:18.272753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517eaf1-667f-4996-997d-0eabf6a94d11 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:18.273069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517eaf1-667f-4996-997d-0eabf6a94d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:18.273344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517eaf1-667f-4996-997d-0eabf6a94d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:18.283797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517eaf1-667f-4996-997d-0eabf6a94d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:18.285360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517eaf1-667f-4996-997d-0eabf6a94d11 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:18.295989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1185/2336] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:39:18.593112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47ba5d-302d-4600-a512-43071219039b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:18.594628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8b47ba5d-302d-4600-a512-43071219039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:18.607312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47ba5d-302d-4600-a512-43071219039b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:18.612577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:18.613459 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:18.613736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:18.614011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] PATH_INFO: `/v3/projects/4fc380f695504800a6debba4c78613be` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:18.615255 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc380f695504800a6debba4c78613be)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:18.617230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:18.666451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc380f695504800a6debba4c78613be', 'request_id': 'req-ba576771-4140-43b5-a789-6e8a4304e08f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:39:18.672727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc380f695504800a6debba4c78613be', 'request_id': 'req-ba576771-4140-43b5-a789-6e8a4304e08f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:39:18.678261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc380f695504800a6debba4c78613be', 'request_id': 'req-ba576771-4140-43b5-a789-6e8a4304e08f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:39:18.693096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] The token cache is being invalidate because project 4fc380f695504800a6debba4c78613be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:39:18.693308 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:39:18.693919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ba576771-4140-43b5-a789-6e8a4304e08f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:39:18.695516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1152/2337] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:39:18 2022] DELETE /identity/v3/projects/4fc380f695504800a6debba4c78613be => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:39:18.700868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3d853-70e0-4acf-93c5-37f4026c353c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:18.701158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3d853-70e0-4acf-93c5-37f4026c353c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:18.701572 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3d853-70e0-4acf-93c5-37f4026c353c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:18.720583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-22a3d853-70e0-4acf-93c5-37f4026c353c None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:39:18.721819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-22a3d853-70e0-4acf-93c5-37f4026c353c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:18.741893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:39:18.787321 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1186/2338] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:39:18.796494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccb0c1-3c32-48ff-9891-3168758f9cf3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:18.796940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccb0c1-3c32-48ff-9891-3168758f9cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:18.797222 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccb0c1-3c32-48ff-9891-3168758f9cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:18.816322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1eccb0c1-3c32-48ff-9891-3168758f9cf3 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:39:18.817552 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1eccb0c1-3c32-48ff-9891-3168758f9cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:18.826601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1153/2339] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:39:18.831964 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3d853-70e0-4acf-93c5-37f4026c353c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:18.847322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3d853-70e0-4acf-93c5-37f4026c353c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:18.854748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9178f73-71dd-418b-b4d4-cea2fb9abe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:18.855675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178f73-71dd-418b-b4d4-cea2fb9abe02 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:18.855990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178f73-71dd-418b-b4d4-cea2fb9abe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:18.856280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9178f73-71dd-418b-b4d4-cea2fb9abe02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:18.857249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9178f73-71dd-418b-b4d4-cea2fb9abe02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:39:18.857561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9178f73-71dd-418b-b4d4-cea2fb9abe02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:18.859725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9178f73-71dd-418b-b4d4-cea2fb9abe02 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:18.871958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1187/2340] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:39:18.876643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccb0c1-3c32-48ff-9891-3168758f9cf3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:18.886076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccb0c1-3c32-48ff-9891-3168758f9cf3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:18.891968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865c058-a6e3-4a0c-a218-465368a432bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:18.892620 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865c058-a6e3-4a0c-a218-465368a432bd admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:18.892901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865c058-a6e3-4a0c-a218-465368a432bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:18.893172 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865c058-a6e3-4a0c-a218-465368a432bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:18.893887 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865c058-a6e3-4a0c-a218-465368a432bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:18.895832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865c058-a6e3-4a0c-a218-465368a432bd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:18.924519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1154/2341] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:39:18 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:39:18.928801 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9178f73-71dd-418b-b4d4-cea2fb9abe02 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:18.939929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9178f73-71dd-418b-b4d4-cea2fb9abe02 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:19.003306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e85d8-a439-4d51-b0cb-626596dcb3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:19.003916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e85d8-a439-4d51-b0cb-626596dcb3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:19.004201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e85d8-a439-4d51-b0cb-626596dcb3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:19.004472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e85d8-a439-4d51-b0cb-626596dcb3c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:19.005175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e85d8-a439-4d51-b0cb-626596dcb3c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:19.007172 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e85d8-a439-4d51-b0cb-626596dcb3c5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:19.030338 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1188/2342] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:39:18 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:39:19.034835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865c058-a6e3-4a0c-a218-465368a432bd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:19.045846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865c058-a6e3-4a0c-a218-465368a432bd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:19.051936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57c4e0-1f3b-4adb-9e51-b9e718f75c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:19.052609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57c4e0-1f3b-4adb-9e51-b9e718f75c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:19.052895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57c4e0-1f3b-4adb-9e51-b9e718f75c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:19.053171 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57c4e0-1f3b-4adb-9e51-b9e718f75c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:19.053909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57c4e0-1f3b-4adb-9e51-b9e718f75c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:19.055575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57c4e0-1f3b-4adb-9e51-b9e718f75c07 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:19.066883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1155/2343] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:39:19.071014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e85d8-a439-4d51-b0cb-626596dcb3c5 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:19.082653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e85d8-a439-4d51-b0cb-626596dcb3c5 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:19.088273 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7bfeb-9724-4ae0-adef-374e8e90e7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:19.088941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7bfeb-9724-4ae0-adef-374e8e90e7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:19.089225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7bfeb-9724-4ae0-adef-374e8e90e7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:19.089504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7bfeb-9724-4ae0-adef-374e8e90e7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:19.090613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7bfeb-9724-4ae0-adef-374e8e90e7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:19.092291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7bfeb-9724-4ae0-adef-374e8e90e7a6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:19.106290 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1189/2344] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:39:19.111659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57c4e0-1f3b-4adb-9e51-b9e718f75c07 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:19.122312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57c4e0-1f3b-4adb-9e51-b9e718f75c07 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:19.128501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593cdce-e536-45df-8062-208cd8d745bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:19.129404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593cdce-e536-45df-8062-208cd8d745bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:19.129689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593cdce-e536-45df-8062-208cd8d745bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:19.129967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593cdce-e536-45df-8062-208cd8d745bc admin admin] PATH_INFO: `/v3/projects/fa6da3e4a87442299d21f3b55dfe3e9f/users/1b2337595cbb40608d6bc485889d8bc6/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:19.146235 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593cdce-e536-45df-8062-208cd8d745bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa6da3e4a87442299d21f3b55dfe3e9f, user_id=1b2337595cbb40608d6bc485889d8bc6, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:19.148990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593cdce-e536-45df-8062-208cd8d745bc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:19.164826 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1156/2345] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:39:19 2022] PUT /identity/v3/projects/fa6da3e4a87442299d21f3b55dfe3e9f/users/1b2337595cbb40608d6bc485889d8bc6/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:39:19.169796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533c0ed-086c-4fe9-bd9f-00ecd7088971 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:19.169951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533c0ed-086c-4fe9-bd9f-00ecd7088971 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:19.170196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533c0ed-086c-4fe9-bd9f-00ecd7088971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:19.195591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9533c0ed-086c-4fe9-bd9f-00ecd7088971 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:39:19.196802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9533c0ed-086c-4fe9-bd9f-00ecd7088971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:19.259073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1190/2346] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:39:19.268432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593cdce-e536-45df-8062-208cd8d745bc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:19.270221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e593cdce-e536-45df-8062-208cd8d745bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:19.293875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:39:19.335713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593cdce-e536-45df-8062-208cd8d745bc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:19.339766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4091-b6d9-4a96-a676-83db4687829b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:19.340772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4091-b6d9-4a96-a676-83db4687829b service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:19.341056 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4091-b6d9-4a96-a676-83db4687829b service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:19.341338 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb4091-b6d9-4a96-a676-83db4687829b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:19.351133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb4091-b6d9-4a96-a676-83db4687829b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:19.352607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb4091-b6d9-4a96-a676-83db4687829b service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:19.362205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1157/2347] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:39:23.452048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19418c92-5c96-43c6-b723-02145fcdbcc1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:23.452663 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19418c92-5c96-43c6-b723-02145fcdbcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:23.452663 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19418c92-5c96-43c6-b723-02145fcdbcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:23.483715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-19418c92-5c96-43c6-b723-02145fcdbcc1 None None] MFA Rules not processed for user `1b2337595cbb40608d6bc485889d8bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:39:23.485309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-19418c92-5c96-43c6-b723-02145fcdbcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:23.503295 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:39:23.558654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1191/2348] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:39:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:39:23.563657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4091-b6d9-4a96-a676-83db4687829b service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:23.575006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb4091-b6d9-4a96-a676-83db4687829b service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:23.580946 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57dc8f-ea5a-4fc1-b4f1-fdcc7887561c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:23.581625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57dc8f-ea5a-4fc1-b4f1-fdcc7887561c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:23.581912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57dc8f-ea5a-4fc1-b4f1-fdcc7887561c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:23.582190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57dc8f-ea5a-4fc1-b4f1-fdcc7887561c admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:23.582933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57dc8f-ea5a-4fc1-b4f1-fdcc7887561c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:23.584868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57dc8f-ea5a-4fc1-b4f1-fdcc7887561c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:23.615668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1158/2349] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:39:23 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:39:23.620136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19418c92-5c96-43c6-b723-02145fcdbcc1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:23.634185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19418c92-5c96-43c6-b723-02145fcdbcc1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:23.693432 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0c96c-d9e1-4a8b-845c-e2410affae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:23.694124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0c96c-d9e1-4a8b-845c-e2410affae47 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:23.694423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0c96c-d9e1-4a8b-845c-e2410affae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:23.694684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0c96c-d9e1-4a8b-845c-e2410affae47 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:23.695428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0c96c-d9e1-4a8b-845c-e2410affae47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:23.697419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0c96c-d9e1-4a8b-845c-e2410affae47 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:23.726139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1192/2350] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:39:23 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:39:23.727963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57dc8f-ea5a-4fc1-b4f1-fdcc7887561c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:23.742537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57dc8f-ea5a-4fc1-b4f1-fdcc7887561c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:23.748566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60363270-0d82-466b-9eec-3eb5c9cf6cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:23.749646 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60363270-0d82-466b-9eec-3eb5c9cf6cfd admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:23.750216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60363270-0d82-466b-9eec-3eb5c9cf6cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:23.750586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60363270-0d82-466b-9eec-3eb5c9cf6cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:23.751564 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60363270-0d82-466b-9eec-3eb5c9cf6cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:23.753596 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60363270-0d82-466b-9eec-3eb5c9cf6cfd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:23.765463 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1159/2351] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:39:23.770046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0c96c-d9e1-4a8b-845c-e2410affae47 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:23.780331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0c96c-d9e1-4a8b-845c-e2410affae47 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:23.785223 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320a41e-6849-4140-af43-71a1ba71c2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:23.786127 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320a41e-6849-4140-af43-71a1ba71c2cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:23.786409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320a41e-6849-4140-af43-71a1ba71c2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:23.786686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1320a41e-6849-4140-af43-71a1ba71c2cb admin admin] PATH_INFO: `/v3/projects/b4362510130a4e99b74fd45c144de17f/users/a0e1a02e9ec142a7a81bc0b3c5f81500/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:23.802258 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320a41e-6849-4140-af43-71a1ba71c2cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4362510130a4e99b74fd45c144de17f, user_id=a0e1a02e9ec142a7a81bc0b3c5f81500, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:23.804957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1320a41e-6849-4140-af43-71a1ba71c2cb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:23.820384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1193/2352] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:39:23 2022] PUT /identity/v3/projects/b4362510130a4e99b74fd45c144de17f/users/a0e1a02e9ec142a7a81bc0b3c5f81500/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:39:23.824677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60363270-0d82-466b-9eec-3eb5c9cf6cfd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:23.838490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60363270-0d82-466b-9eec-3eb5c9cf6cfd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:23.916828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646d8b5-5a76-48d3-a300-4cb34f01eea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:23.917508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646d8b5-5a76-48d3-a300-4cb34f01eea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:23.917792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646d8b5-5a76-48d3-a300-4cb34f01eea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:23.918069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646d8b5-5a76-48d3-a300-4cb34f01eea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:23.918846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646d8b5-5a76-48d3-a300-4cb34f01eea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:23.920465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646d8b5-5a76-48d3-a300-4cb34f01eea8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:23.930609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1160/2353] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:39:23.935227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320a41e-6849-4140-af43-71a1ba71c2cb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:23.949213 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1320a41e-6849-4140-af43-71a1ba71c2cb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:23.959458 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d6a91-dd46-485d-a7b9-49bab8536ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:23.960386 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d6a91-dd46-485d-a7b9-49bab8536ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:23.960836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d6a91-dd46-485d-a7b9-49bab8536ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:23.961018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d6a91-dd46-485d-a7b9-49bab8536ab8 admin admin] PATH_INFO: `/v3/projects/b4362510130a4e99b74fd45c144de17f/users/a0e1a02e9ec142a7a81bc0b3c5f81500/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:23.963241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d6a91-dd46-485d-a7b9-49bab8536ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4362510130a4e99b74fd45c144de17f, user_id=a0e1a02e9ec142a7a81bc0b3c5f81500, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:23.966273 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d6a91-dd46-485d-a7b9-49bab8536ab8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:23.977888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1194/2354] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:39:23 2022] PUT /identity/v3/projects/b4362510130a4e99b74fd45c144de17f/users/a0e1a02e9ec142a7a81bc0b3c5f81500/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:39:28.699481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a361f7-1d58-48f5-bacd-2d90c45e968c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:28.699999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a361f7-1d58-48f5-bacd-2d90c45e968c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:28.700106 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a361f7-1d58-48f5-bacd-2d90c45e968c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:28.733726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-20a361f7-1d58-48f5-bacd-2d90c45e968c None None] MFA Rules not processed for user `a0e1a02e9ec142a7a81bc0b3c5f81500`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:39:28.735910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-20a361f7-1d58-48f5-bacd-2d90c45e968c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:28.807945 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1161/2355] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:39:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:39:28.822487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d6a91-dd46-485d-a7b9-49bab8536ab8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:39:28.824521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2c0d6a91-dd46-485d-a7b9-49bab8536ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:39:28.903798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:39:28.940638 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d6a91-dd46-485d-a7b9-49bab8536ab8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:39:28.954051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a49c36-a4e6-44b1-809d-0a82fff206a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:39:28.954836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a49c36-a4e6-44b1-809d-0a82fff206a4 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:39:28.955130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a49c36-a4e6-44b1-809d-0a82fff206a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:39:28.955399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a49c36-a4e6-44b1-809d-0a82fff206a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:39:29.030555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a49c36-a4e6-44b1-809d-0a82fff206a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:39:29.032460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a49c36-a4e6-44b1-809d-0a82fff206a4 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:39:29.043439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1195/2356] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:39:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:39:38.805066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:39:39.038158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=36 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:40:08.812997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a361f7-1d58-48f5-bacd-2d90c45e968c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:40:08.814718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-20a361f7-1d58-48f5-bacd-2d90c45e968c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:08.875342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a361f7-1d58-48f5-bacd-2d90c45e968c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:40:08.879208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d642c83-4000-44ce-bc04-fda9326c61c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:40:08.879861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d642c83-4000-44ce-bc04-fda9326c61c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:40:08.880136 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d642c83-4000-44ce-bc04-fda9326c61c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:40:08.880404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d642c83-4000-44ce-bc04-fda9326c61c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:40:08.882323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2d642c83-4000-44ce-bc04-fda9326c61c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:08.896005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d642c83-4000-44ce-bc04-fda9326c61c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:40:08.897439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d642c83-4000-44ce-bc04-fda9326c61c6 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:40:08.908471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1162/2357] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:40:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:40:11.044666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682543a-e951-4582-ac89-b6c2852117ee None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:40:11.045206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682543a-e951-4582-ac89-b6c2852117ee None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:40:11.045206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682543a-e951-4582-ac89-b6c2852117ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:40:11.063942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5682543a-e951-4582-ac89-b6c2852117ee None None] MFA Rules not processed for user `1b2337595cbb40608d6bc485889d8bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:40:11.065319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5682543a-e951-4582-ac89-b6c2852117ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:11.073240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1196/2358] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:40:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:40:11.114236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d642c83-4000-44ce-bc04-fda9326c61c6 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:40:11.115966 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2d642c83-4000-44ce-bc04-fda9326c61c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:11.127441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d642c83-4000-44ce-bc04-fda9326c61c6 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:40:11.131657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd5f85-6dad-40b1-b3e5-07d5c4e5e6e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:40:11.132301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd5f85-6dad-40b1-b3e5-07d5c4e5e6e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:40:11.132576 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd5f85-6dad-40b1-b3e5-07d5c4e5e6e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:40:11.132847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd5f85-6dad-40b1-b3e5-07d5c4e5e6e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:40:11.134956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-48fd5f85-6dad-40b1-b3e5-07d5c4e5e6e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:11.149313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd5f85-6dad-40b1-b3e5-07d5c4e5e6e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:40:11.151687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd5f85-6dad-40b1-b3e5-07d5c4e5e6e7 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:40:11.165809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1163/2359] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:40:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:40:21.058511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:40:21.161176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=32 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:40:25.438570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682543a-e951-4582-ac89-b6c2852117ee None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:40:25.454847 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682543a-e951-4582-ac89-b6c2852117ee None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:40:25.462659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcab6c0-b568-46c5-b0b5-124a2d88fbd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:40:25.463380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcab6c0-b568-46c5-b0b5-124a2d88fbd3 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:40:25.463651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcab6c0-b568-46c5-b0b5-124a2d88fbd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:40:25.463920 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcab6c0-b568-46c5-b0b5-124a2d88fbd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:40:25.466410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3dcab6c0-b568-46c5-b0b5-124a2d88fbd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:25.533281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcab6c0-b568-46c5-b0b5-124a2d88fbd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:40:25.534933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcab6c0-b568-46c5-b0b5-124a2d88fbd3 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:40:25.544719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1197/2360] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:40:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:40:35.539906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=28 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:40:38.190389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd5f85-6dad-40b1-b3e5-07d5c4e5e6e7 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:40:38.202146 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd5f85-6dad-40b1-b3e5-07d5c4e5e6e7 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:40:38.209373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdf80c-c457-40b7-88c4-3da02d8e1593 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:40:38.210017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf80c-c457-40b7-88c4-3da02d8e1593 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:40:38.210267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf80c-c457-40b7-88c4-3da02d8e1593 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:40:38.210559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdf80c-c457-40b7-88c4-3da02d8e1593 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:40:38.212815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1cfdf80c-c457-40b7-88c4-3da02d8e1593 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:38.280070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdf80c-c457-40b7-88c4-3da02d8e1593 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:40:38.281761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdf80c-c457-40b7-88c4-3da02d8e1593 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:40:38.292721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1164/2361] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:40:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:40:42.966081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcab6c0-b568-46c5-b0b5-124a2d88fbd3 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:40:42.967652 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3dcab6c0-b568-46c5-b0b5-124a2d88fbd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:43.026198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcab6c0-b568-46c5-b0b5-124a2d88fbd3 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:40:43.030418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca545bb-d37b-4310-97c5-e9514a886aa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:40:43.031087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca545bb-d37b-4310-97c5-e9514a886aa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:40:43.031373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca545bb-d37b-4310-97c5-e9514a886aa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:40:43.031896 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca545bb-d37b-4310-97c5-e9514a886aa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:40:43.041001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca545bb-d37b-4310-97c5-e9514a886aa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:40:43.042442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca545bb-d37b-4310-97c5-e9514a886aa8 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:40:43.052394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1198/2362] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:40:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:40:48.287121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=28 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:40:48.619340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec9d67-eedc-49b4-9022-431f4e14debb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:40:48.619632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec9d67-eedc-49b4-9022-431f4e14debb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:40:48.619908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec9d67-eedc-49b4-9022-431f4e14debb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:40:48.638097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-eaec9d67-eedc-49b4-9022-431f4e14debb None None] MFA Rules not processed for user `bccf002d9da34d1dad0ee71b5b7bb1c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:40:48.639330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eaec9d67-eedc-49b4-9022-431f4e14debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:48.647107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1165/2363] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:40:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:40:48.981115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca545bb-d37b-4310-97c5-e9514a886aa8 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:40:48.990810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca545bb-d37b-4310-97c5-e9514a886aa8 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:40:48.995002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7300f5-6513-47ab-9a72-051863f497f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:40:48.995642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7300f5-6513-47ab-9a72-051863f497f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:40:48.995926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7300f5-6513-47ab-9a72-051863f497f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:40:48.996199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7300f5-6513-47ab-9a72-051863f497f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:40:48.998303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7c7300f5-6513-47ab-9a72-051863f497f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:49.008314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7300f5-6513-47ab-9a72-051863f497f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:40:49.009753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7300f5-6513-47ab-9a72-051863f497f7 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:40:49.018255 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1199/2364] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:40:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:40:49.365787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec9d67-eedc-49b4-9022-431f4e14debb None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:40:49.367828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eaec9d67-eedc-49b4-9022-431f4e14debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:49.445513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec9d67-eedc-49b4-9022-431f4e14debb None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:40:49.450659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929b521-b1f8-47c3-afad-8108922332f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:40:49.451387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929b521-b1f8-47c3-afad-8108922332f4 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:40:49.451640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929b521-b1f8-47c3-afad-8108922332f4 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:40:49.451948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7929b521-b1f8-47c3-afad-8108922332f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:40:49.454071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7929b521-b1f8-47c3-afad-8108922332f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:49.466869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929b521-b1f8-47c3-afad-8108922332f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:40:49.468482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7929b521-b1f8-47c3-afad-8108922332f4 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:40:49.478836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1166/2365] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:40:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:40:51.968023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7300f5-6513-47ab-9a72-051863f497f7 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:40:51.978591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7300f5-6513-47ab-9a72-051863f497f7 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:40:51.983138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682029ed-427c-41ef-8a4b-9ef886c6c0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:40:51.983834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682029ed-427c-41ef-8a4b-9ef886c6c0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:40:51.984112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682029ed-427c-41ef-8a4b-9ef886c6c0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:40:51.984381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682029ed-427c-41ef-8a4b-9ef886c6c0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:40:51.986330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-682029ed-427c-41ef-8a4b-9ef886c6c0a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:51.998911 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682029ed-427c-41ef-8a4b-9ef886c6c0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:40:52.000707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682029ed-427c-41ef-8a4b-9ef886c6c0a1 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:40:52.010397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1200/2366] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:40:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:40:52.640842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929b521-b1f8-47c3-afad-8108922332f4 service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:40:52.642343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7929b521-b1f8-47c3-afad-8108922332f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:52.666276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7929b521-b1f8-47c3-afad-8108922332f4 service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:40:52.678758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524b219-d15d-44f4-8655-0a213473349e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:40:52.679388 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524b219-d15d-44f4-8655-0a213473349e service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:40:52.679672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524b219-d15d-44f4-8655-0a213473349e service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:40:52.680011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524b219-d15d-44f4-8655-0a213473349e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:40:52.682809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a524b219-d15d-44f4-8655-0a213473349e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:40:52.696741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524b219-d15d-44f4-8655-0a213473349e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:40:52.698501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524b219-d15d-44f4-8655-0a213473349e service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:40:52.711420 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1167/2367] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:40:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:41:02.006003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=38 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:41:02.706230 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=41 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:41:04.812368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0fbb-d43b-450e-be6a-1d4a324ba0c3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:04.812945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0fbb-d43b-450e-be6a-1d4a324ba0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:04.812945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f0fbb-d43b-450e-be6a-1d4a324ba0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:04.832905 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7b3f0fbb-d43b-450e-be6a-1d4a324ba0c3 None None] MFA Rules not processed for user `e3be3323efb945f1ad6537bbc9eba7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:41:04.834136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7b3f0fbb-d43b-450e-be6a-1d4a324ba0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:41:04.898132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1201/2368] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:41:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:41:05.143761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524b219-d15d-44f4-8655-0a213473349e service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:41:05.153709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524b219-d15d-44f4-8655-0a213473349e service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:41:05.157377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f39fa6-9301-401f-92bc-c4d0b5880581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:41:05.158028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f39fa6-9301-401f-92bc-c4d0b5880581 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:05.158313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f39fa6-9301-401f-92bc-c4d0b5880581 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:05.158719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f39fa6-9301-401f-92bc-c4d0b5880581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:05.160848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c6f39fa6-9301-401f-92bc-c4d0b5880581 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:41:05.172322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f39fa6-9301-401f-92bc-c4d0b5880581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:41:05.174728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f39fa6-9301-401f-92bc-c4d0b5880581 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:41:05.183432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1168/2369] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:41:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:41:08.725760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f0fbb-d43b-450e-be6a-1d4a324ba0c3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:41:08.739726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f0fbb-d43b-450e-be6a-1d4a324ba0c3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:41:08.746408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcebba-7131-41b8-8cff-4841643d0e91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:41:08.747087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcebba-7131-41b8-8cff-4841643d0e91 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:08.747363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcebba-7131-41b8-8cff-4841643d0e91 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:08.747634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcebba-7131-41b8-8cff-4841643d0e91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:08.750242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f9fcebba-7131-41b8-8cff-4841643d0e91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:41:08.763016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fcebba-7131-41b8-8cff-4841643d0e91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:41:08.764800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fcebba-7131-41b8-8cff-4841643d0e91 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:41:08.777218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1202/2370] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:41:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:41:15.179911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:41:18.774832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=31 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:41:24.307589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f39fa6-9301-401f-92bc-c4d0b5880581 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:41:24.322885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f39fa6-9301-401f-92bc-c4d0b5880581 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:41:24.329747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724dcfb-6621-4ac7-8b13-16b02e04d30b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:41:24.330428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724dcfb-6621-4ac7-8b13-16b02e04d30b service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:24.330716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724dcfb-6621-4ac7-8b13-16b02e04d30b service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:24.331005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724dcfb-6621-4ac7-8b13-16b02e04d30b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:24.333223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a724dcfb-6621-4ac7-8b13-16b02e04d30b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:41:24.347847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724dcfb-6621-4ac7-8b13-16b02e04d30b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:41:24.349458 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724dcfb-6621-4ac7-8b13-16b02e04d30b service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:41:24.360762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1169/2371] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:41:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:41:31.938742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48a6cd-3f73-4f6b-86dc-c5e9b7abfd89 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:31.939582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48a6cd-3f73-4f6b-86dc-c5e9b7abfd89 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:31.939934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48a6cd-3f73-4f6b-86dc-c5e9b7abfd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:32.007118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7e48a6cd-3f73-4f6b-86dc-c5e9b7abfd89 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:41:32.008249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7e48a6cd-3f73-4f6b-86dc-c5e9b7abfd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:41:32.095891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1203/2372] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:41:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:41:32.145085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724dcfb-6621-4ac7-8b13-16b02e04d30b service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:41:32.146584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a724dcfb-6621-4ac7-8b13-16b02e04d30b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:41:32.244118 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724dcfb-6621-4ac7-8b13-16b02e04d30b service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:41:32.250875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f4160-1a31-4fd8-a497-1847a5bc4c15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:41:32.251773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f4160-1a31-4fd8-a497-1847a5bc4c15 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:32.252078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f4160-1a31-4fd8-a497-1847a5bc4c15 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:32.252437 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f4160-1a31-4fd8-a497-1847a5bc4c15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:32.267400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f4160-1a31-4fd8-a497-1847a5bc4c15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:41:32.269705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f4160-1a31-4fd8-a497-1847a5bc4c15 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:41:32.281085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1170/2373] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:41:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:41:42.092436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=44 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:41:42.277290 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=47 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:41:51.031962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf55d7-0d2a-427a-97d2-d16149bc7232 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:51.032565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf55d7-0d2a-427a-97d2-d16149bc7232 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:51.032565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf55d7-0d2a-427a-97d2-d16149bc7232 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:51.033883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1204/2374] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:41:51.040482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d70fe9-3c5c-43fd-8b85-fdb79b2174a4 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:51.041063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d70fe9-3c5c-43fd-8b85-fdb79b2174a4 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:51.041063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d70fe9-3c5c-43fd-8b85-fdb79b2174a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:51.080436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-85d70fe9-3c5c-43fd-8b85-fdb79b2174a4 None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:41:51.081686 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-85d70fe9-3c5c-43fd-8b85-fdb79b2174a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:41:51.092362 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1171/2375] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:41:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:41:51.105282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf55d7-0d2a-427a-97d2-d16149bc7232 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:41:51.107012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7bcf55d7-0d2a-427a-97d2-d16149bc7232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:41:51.194026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf55d7-0d2a-427a-97d2-d16149bc7232 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:41:51.197861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e1a7b9-374e-4a14-b222-72a4d45d7db1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd09453b5db645d7b03522f91effb8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:41:51.198530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1a7b9-374e-4a14-b222-72a4d45d7db1 service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:51.198846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1a7b9-374e-4a14-b222-72a4d45d7db1 service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:51.199121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1a7b9-374e-4a14-b222-72a4d45d7db1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:51.213518 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1a7b9-374e-4a14-b222-72a4d45d7db1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:41:51.215053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e1a7b9-374e-4a14-b222-72a4d45d7db1 service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:41:51.227703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1205/2376] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:41:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:41:51.297640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd7d57-8ffe-4d26-8351-a256fdc57aaa None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:51.298020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd7d57-8ffe-4d26-8351-a256fdc57aaa None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:51.298360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd7d57-8ffe-4d26-8351-a256fdc57aaa None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:51.299915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1172/2377] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:41:51.305180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ea91f-3504-4fef-89e0-c207cb6d72a0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:51.305754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ea91f-3504-4fef-89e0-c207cb6d72a0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:51.306232 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ea91f-3504-4fef-89e0-c207cb6d72a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:51.326786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-756ea91f-3504-4fef-89e0-c207cb6d72a0 None None] MFA Rules not processed for user `49a1439029fa4b719bc114c3f59e0050`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:41:51.328029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-756ea91f-3504-4fef-89e0-c207cb6d72a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:41:51.335433 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1206/2378] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:41:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:41:51.347495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd7d57-8ffe-4d26-8351-a256fdc57aaa None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:41:51.359246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd7d57-8ffe-4d26-8351-a256fdc57aaa None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:41:51.362855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd8c81-4830-4073-b165-4896468bf7d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:41:51.363489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd8c81-4830-4073-b165-4896468bf7d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:51.363764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd8c81-4830-4073-b165-4896468bf7d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:51.364033 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd8c81-4830-4073-b165-4896468bf7d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:51.371268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd8c81-4830-4073-b165-4896468bf7d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:41:51.372709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd8c81-4830-4073-b165-4896468bf7d4 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:41:51.380879 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1173/2379] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:41:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:41:51.473597 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84000ccb-fff5-4cce-b234-d3fb1f766882 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:51.473886 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84000ccb-fff5-4cce-b234-d3fb1f766882 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:51.474160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84000ccb-fff5-4cce-b234-d3fb1f766882 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:51.475599 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1207/2380] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:41:51.480913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99028c-0190-41a3-9d92-bbc2a7bc534a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:51.481276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99028c-0190-41a3-9d92-bbc2a7bc534a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:51.481561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99028c-0190-41a3-9d92-bbc2a7bc534a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:51.501904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9c99028c-0190-41a3-9d92-bbc2a7bc534a None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:41:51.503088 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9c99028c-0190-41a3-9d92-bbc2a7bc534a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:41:51.513546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1174/2381] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:41:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:41:51.524282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84000ccb-fff5-4cce-b234-d3fb1f766882 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:41:51.535843 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84000ccb-fff5-4cce-b234-d3fb1f766882 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:41:51.540365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510aa42-c3d3-4905-b857-885ae58a364d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:41:51.541027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510aa42-c3d3-4905-b857-885ae58a364d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:51.541281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510aa42-c3d3-4905-b857-885ae58a364d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:51.541556 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510aa42-c3d3-4905-b857-885ae58a364d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:51.552676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510aa42-c3d3-4905-b857-885ae58a364d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:41:51.554899 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510aa42-c3d3-4905-b857-885ae58a364d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:41:51.573854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1208/2382] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:41:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:41:55.158677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40bf2a-ec7f-4b0d-9d32-925f6d4cb9e7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:55.159199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40bf2a-ec7f-4b0d-9d32-925f6d4cb9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:55.159336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40bf2a-ec7f-4b0d-9d32-925f6d4cb9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:55.177781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5c40bf2a-ec7f-4b0d-9d32-925f6d4cb9e7 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:41:55.179110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5c40bf2a-ec7f-4b0d-9d32-925f6d4cb9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:41:55.187384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1175/2383] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:41:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:41:55.199742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510aa42-c3d3-4905-b857-885ae58a364d service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:41:55.211685 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510aa42-c3d3-4905-b857-885ae58a364d service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:41:55.218082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49b5f3-bb5e-4dc1-ab6e-8e5726ce61a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:41:55.218741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b5f3-bb5e-4dc1-ab6e-8e5726ce61a1 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:41:55.219010 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b5f3-bb5e-4dc1-ab6e-8e5726ce61a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:41:55.219305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49b5f3-bb5e-4dc1-ab6e-8e5726ce61a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:41:55.227907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49b5f3-bb5e-4dc1-ab6e-8e5726ce61a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:41:55.229501 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49b5f3-bb5e-4dc1-ab6e-8e5726ce61a1 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:41:55.237487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1209/2384] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:41:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:42:04.258486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40bf2a-ec7f-4b0d-9d32-925f6d4cb9e7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:04.273964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40bf2a-ec7f-4b0d-9d32-925f6d4cb9e7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:04.280492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4877b7f-79e5-439d-b548-c42e4e06153f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:04.281157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4877b7f-79e5-439d-b548-c42e4e06153f service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:04.281527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4877b7f-79e5-439d-b548-c42e4e06153f service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:04.281822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4877b7f-79e5-439d-b548-c42e4e06153f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:04.284091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b4877b7f-79e5-439d-b548-c42e4e06153f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:04.344983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4877b7f-79e5-439d-b548-c42e4e06153f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:04.347071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4877b7f-79e5-439d-b548-c42e4e06153f service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:04.357168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1176/2385] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:42:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:42:05.234664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=50 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:42:14.352621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=49 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:42:29.167790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49b5f3-bb5e-4dc1-ab6e-8e5726ce61a1 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:29.169598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0a49b5f3-bb5e-4dc1-ab6e-8e5726ce61a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:29.185759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49b5f3-bb5e-4dc1-ab6e-8e5726ce61a1 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:29.192807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b5865-76a7-40c5-97c2-fe0d1f5c424e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:29.193452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5865-76a7-40c5-97c2-fe0d1f5c424e service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:29.193960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5865-76a7-40c5-97c2-fe0d1f5c424e service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:29.194410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b5865-76a7-40c5-97c2-fe0d1f5c424e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:29.196522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-515b5865-76a7-40c5-97c2-fe0d1f5c424e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:29.209962 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b5865-76a7-40c5-97c2-fe0d1f5c424e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:29.211594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b5865-76a7-40c5-97c2-fe0d1f5c424e service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:29.229196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1210/2386] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:42:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:42:29.232610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4877b7f-79e5-439d-b548-c42e4e06153f service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:29.234234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b4877b7f-79e5-439d-b548-c42e4e06153f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:29.308109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4877b7f-79e5-439d-b548-c42e4e06153f service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:29.313033 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb83b9e-acdc-4b74-8bc0-3fcddeaf0055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:29.313982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83b9e-acdc-4b74-8bc0-3fcddeaf0055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:29.314265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83b9e-acdc-4b74-8bc0-3fcddeaf0055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:29.314577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb83b9e-acdc-4b74-8bc0-3fcddeaf0055 admin admin] PATH_INFO: `/v3/users/e3be3323efb945f1ad6537bbc9eba7b0` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:29.315806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb83b9e-acdc-4b74-8bc0-3fcddeaf0055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3be3323efb945f1ad6537bbc9eba7b0)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:29.317782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb83b9e-acdc-4b74-8bc0-3fcddeaf0055 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:29.378208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-edb83b9e-acdc-4b74-8bc0-3fcddeaf0055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3be3323efb945f1ad6537bbc9eba7b0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:42:29.379005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-edb83b9e-acdc-4b74-8bc0-3fcddeaf0055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3be3323efb945f1ad6537bbc9eba7b0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:42:29.391165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-edb83b9e-acdc-4b74-8bc0-3fcddeaf0055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3be3323efb945f1ad6537bbc9eba7b0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:42:29.391853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-edb83b9e-acdc-4b74-8bc0-3fcddeaf0055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3be3323efb945f1ad6537bbc9eba7b0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:42:29.400597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-edb83b9e-acdc-4b74-8bc0-3fcddeaf0055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3be3323efb945f1ad6537bbc9eba7b0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:42:29.413124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1177/2387] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:42:29 2022] DELETE /identity/v3/users/e3be3323efb945f1ad6537bbc9eba7b0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:42:29.597837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b5865-76a7-40c5-97c2-fe0d1f5c424e service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:29.621714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b5865-76a7-40c5-97c2-fe0d1f5c424e service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:29.680671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:29.681534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:29.681811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:29.682076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] PATH_INFO: `/v3/projects/d1cc844419c14acd9a9e7dfb4233841d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:29.683305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1cc844419c14acd9a9e7dfb4233841d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:29.685283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:29.737121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1cc844419c14acd9a9e7dfb4233841d', 'request_id': 'req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:42:29.743328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1cc844419c14acd9a9e7dfb4233841d', 'request_id': 'req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:42:29.750365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1cc844419c14acd9a9e7dfb4233841d', 'request_id': 'req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:42:29.769111 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] The token cache is being invalidate because project d1cc844419c14acd9a9e7dfb4233841d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:42:29.769409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:42:29.770145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-65b3f9c6-794c-4906-8e67-df0c7b6ad3d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:42:29.771822 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1211/2388] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:42:29 2022] DELETE /identity/v3/projects/d1cc844419c14acd9a9e7dfb4233841d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:42:29.777269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2bd2-a7ef-4f8a-bd67-38341f2b85c5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:29.777558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2bd2-a7ef-4f8a-bd67-38341f2b85c5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:29.777836 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2bd2-a7ef-4f8a-bd67-38341f2b85c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:29.802387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-172c2bd2-a7ef-4f8a-bd67-38341f2b85c5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:42:29.803618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-172c2bd2-a7ef-4f8a-bd67-38341f2b85c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:29.864435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1178/2389] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:42:29.872759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d92f7-7478-4fe1-8369-61601cf43a8f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:29.873063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d92f7-7478-4fe1-8369-61601cf43a8f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:29.873328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d92f7-7478-4fe1-8369-61601cf43a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:29.892084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-3e0d92f7-7478-4fe1-8369-61601cf43a8f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:42:29.893271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3e0d92f7-7478-4fe1-8369-61601cf43a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:29.905755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1212/2390] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:42:29.910127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c2bd2-a7ef-4f8a-bd67-38341f2b85c5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:29.920016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c2bd2-a7ef-4f8a-bd67-38341f2b85c5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:29.925014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedcff5-a8e5-4c69-b6ed-46d717783e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:29.925675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedcff5-a8e5-4c69-b6ed-46d717783e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:29.925952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedcff5-a8e5-4c69-b6ed-46d717783e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:29.926221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedcff5-a8e5-4c69-b6ed-46d717783e14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:29.926985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dedcff5-a8e5-4c69-b6ed-46d717783e14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:42:29.927287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dedcff5-a8e5-4c69-b6ed-46d717783e14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:29.928871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dedcff5-a8e5-4c69-b6ed-46d717783e14 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:29.940870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1179/2391] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:42:29.945589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d92f7-7478-4fe1-8369-61601cf43a8f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:29.956029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d92f7-7478-4fe1-8369-61601cf43a8f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:29.961948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d8ac9-78c0-4e2c-89de-ab5082c6c3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:29.962573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d8ac9-78c0-4e2c-89de-ab5082c6c3de admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:29.962884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d8ac9-78c0-4e2c-89de-ab5082c6c3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:29.963156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d8ac9-78c0-4e2c-89de-ab5082c6c3de admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:29.963848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d8ac9-78c0-4e2c-89de-ab5082c6c3de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:29.966462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d8ac9-78c0-4e2c-89de-ab5082c6c3de admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:30.006150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1213/2392] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:42:29 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:42:30.010640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedcff5-a8e5-4c69-b6ed-46d717783e14 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:30.021602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedcff5-a8e5-4c69-b6ed-46d717783e14 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:30.025690 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:42:30.082024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405401fd-ee0f-47d3-9b2c-d9b1284f4151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:30.082715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405401fd-ee0f-47d3-9b2c-d9b1284f4151 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:30.083012 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405401fd-ee0f-47d3-9b2c-d9b1284f4151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:30.083287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405401fd-ee0f-47d3-9b2c-d9b1284f4151 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:30.083980 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405401fd-ee0f-47d3-9b2c-d9b1284f4151 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:30.085897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405401fd-ee0f-47d3-9b2c-d9b1284f4151 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:30.111541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1180/2393] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:42:30 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:42:30.112761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d8ac9-78c0-4e2c-89de-ab5082c6c3de admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:30.124499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d8ac9-78c0-4e2c-89de-ab5082c6c3de admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:30.129892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0217781-1c13-45af-ae43-0be0e8274251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:30.130731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0217781-1c13-45af-ae43-0be0e8274251 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:30.130980 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0217781-1c13-45af-ae43-0be0e8274251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:30.131271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0217781-1c13-45af-ae43-0be0e8274251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:30.132037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0217781-1c13-45af-ae43-0be0e8274251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:30.133717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0217781-1c13-45af-ae43-0be0e8274251 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:30.144666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1214/2394] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:42:30.148784 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405401fd-ee0f-47d3-9b2c-d9b1284f4151 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:30.159274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405401fd-ee0f-47d3-9b2c-d9b1284f4151 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:30.164015 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1ff0a-9c79-414c-816f-a43611c0e1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:30.164687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1ff0a-9c79-414c-816f-a43611c0e1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:30.164972 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1ff0a-9c79-414c-816f-a43611c0e1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:30.165252 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1ff0a-9c79-414c-816f-a43611c0e1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:30.165984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1ff0a-9c79-414c-816f-a43611c0e1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:30.167621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1ff0a-9c79-414c-816f-a43611c0e1c1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:30.178488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1181/2395] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:42:30.182621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0217781-1c13-45af-ae43-0be0e8274251 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:30.193031 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0217781-1c13-45af-ae43-0be0e8274251 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:30.198112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176b208-706c-4f3f-bf1b-2ce8b3b6eaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:30.199190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176b208-706c-4f3f-bf1b-2ce8b3b6eaa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:30.199523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176b208-706c-4f3f-bf1b-2ce8b3b6eaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:30.199800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176b208-706c-4f3f-bf1b-2ce8b3b6eaa8 admin admin] PATH_INFO: `/v3/projects/40b4ce1d85294063aea6df4b9638c90f/users/215c8b04d6294311aeb1124b9139c7fc/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:30.216028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176b208-706c-4f3f-bf1b-2ce8b3b6eaa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b4ce1d85294063aea6df4b9638c90f, user_id=215c8b04d6294311aeb1124b9139c7fc, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:30.218837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176b208-706c-4f3f-bf1b-2ce8b3b6eaa8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:30.233403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1215/2396] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:42:30 2022] PUT /identity/v3/projects/40b4ce1d85294063aea6df4b9638c90f/users/215c8b04d6294311aeb1124b9139c7fc/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:42:30.238532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ce06-cce0-4272-9379-388ba173b4c1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:30.239060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ce06-cce0-4272-9379-388ba173b4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:30.239285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8ce06-cce0-4272-9379-388ba173b4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:30.264205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-74e8ce06-cce0-4272-9379-388ba173b4c1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:42:30.264205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-74e8ce06-cce0-4272-9379-388ba173b4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:30.329434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1182/2397] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:42:30.339702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176b208-706c-4f3f-bf1b-2ce8b3b6eaa8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:30.342101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6176b208-706c-4f3f-bf1b-2ce8b3b6eaa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:30.349494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:42:30.407696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176b208-706c-4f3f-bf1b-2ce8b3b6eaa8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:30.411313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d98b94-c73d-449c-9a98-b5f665d570f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:30.411986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d98b94-c73d-449c-9a98-b5f665d570f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:30.412262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d98b94-c73d-449c-9a98-b5f665d570f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:30.412536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d98b94-c73d-449c-9a98-b5f665d570f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:30.424519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d98b94-c73d-449c-9a98-b5f665d570f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:30.425957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d98b94-c73d-449c-9a98-b5f665d570f8 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:30.435628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1216/2398] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:42:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:42:34.332390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591b19b-3b21-448c-84c3-c5a37aa40864 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:34.332910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591b19b-3b21-448c-84c3-c5a37aa40864 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:34.332992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d591b19b-3b21-448c-84c3-c5a37aa40864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:34.352863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d591b19b-3b21-448c-84c3-c5a37aa40864 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:42:34.353984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d591b19b-3b21-448c-84c3-c5a37aa40864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:34.427322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1183/2399] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jun 29 08:42:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:42:34.467827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d98b94-c73d-449c-9a98-b5f665d570f8 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:34.470122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-79d98b94-c73d-449c-9a98-b5f665d570f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:34.576035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d98b94-c73d-449c-9a98-b5f665d570f8 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:34.583598 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f637e5-9285-4a9d-bb42-f60befff2559 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:34.584507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f637e5-9285-4a9d-bb42-f60befff2559 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:34.584940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f637e5-9285-4a9d-bb42-f60befff2559 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:34.585378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f637e5-9285-4a9d-bb42-f60befff2559 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:34.595610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f637e5-9285-4a9d-bb42-f60befff2559 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:34.597624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f637e5-9285-4a9d-bb42-f60befff2559 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:34.607622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1217/2400] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:42:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:42:35.056914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a09f-5e71-4633-af16-e3a42dfe0003 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:35.059348 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a09f-5e71-4633-af16-e3a42dfe0003 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:35.059627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a09f-5e71-4633-af16-e3a42dfe0003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:35.094193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f567a09f-5e71-4633-af16-e3a42dfe0003 None None] MFA Rules not processed for user `215c8b04d6294311aeb1124b9139c7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:42:35.095445 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f567a09f-5e71-4633-af16-e3a42dfe0003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:35.124363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:42:35.177989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1184/2401] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:42:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:42:35.182333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f637e5-9285-4a9d-bb42-f60befff2559 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:35.200708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f637e5-9285-4a9d-bb42-f60befff2559 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:35.203160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142432e5-8831-4b85-8fb5-954ca39bb3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:35.203837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142432e5-8831-4b85-8fb5-954ca39bb3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:35.204120 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142432e5-8831-4b85-8fb5-954ca39bb3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:35.204397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142432e5-8831-4b85-8fb5-954ca39bb3e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:35.205100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142432e5-8831-4b85-8fb5-954ca39bb3e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:35.207141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142432e5-8831-4b85-8fb5-954ca39bb3e6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:35.245880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1218/2402] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:42:35 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:42:35.247875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a09f-5e71-4633-af16-e3a42dfe0003 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:35.258311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a09f-5e71-4633-af16-e3a42dfe0003 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:35.326225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5df59-58dc-4936-8db2-89929b36e0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:35.327034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5df59-58dc-4936-8db2-89929b36e0ff admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:35.327315 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5df59-58dc-4936-8db2-89929b36e0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:35.327625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5df59-58dc-4936-8db2-89929b36e0ff admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:35.328357 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5df59-58dc-4936-8db2-89929b36e0ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:35.331547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5df59-58dc-4936-8db2-89929b36e0ff admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:35.381271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1185/2403] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:42:35 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:42:35.385190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142432e5-8831-4b85-8fb5-954ca39bb3e6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:35.396608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142432e5-8831-4b85-8fb5-954ca39bb3e6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:35.401437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f7a0d-7049-4af1-adbb-533acc50621a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:35.402120 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f7a0d-7049-4af1-adbb-533acc50621a admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:35.402404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f7a0d-7049-4af1-adbb-533acc50621a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:35.402680 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f7a0d-7049-4af1-adbb-533acc50621a admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:35.403477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f7a0d-7049-4af1-adbb-533acc50621a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:35.405289 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f7a0d-7049-4af1-adbb-533acc50621a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:35.417244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1219/2404] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:42:35.422370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5df59-58dc-4936-8db2-89929b36e0ff admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:35.433566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5df59-58dc-4936-8db2-89929b36e0ff admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:35.438644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b963d-6adf-40c7-8d85-4ba319e7c151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:35.439592 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b963d-6adf-40c7-8d85-4ba319e7c151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:35.439871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b963d-6adf-40c7-8d85-4ba319e7c151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:35.440152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b963d-6adf-40c7-8d85-4ba319e7c151 admin admin] PATH_INFO: `/v3/projects/8343434e69cf432a8c029e364416047f/users/5b0ddeea1bbb402590e66e5e25f3722d/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:35.455985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b963d-6adf-40c7-8d85-4ba319e7c151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8343434e69cf432a8c029e364416047f, user_id=5b0ddeea1bbb402590e66e5e25f3722d, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:35.458943 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b963d-6adf-40c7-8d85-4ba319e7c151 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:35.478167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1186/2405] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:42:35 2022] PUT /identity/v3/projects/8343434e69cf432a8c029e364416047f/users/5b0ddeea1bbb402590e66e5e25f3722d/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:42:35.482238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f7a0d-7049-4af1-adbb-533acc50621a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:35.492557 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f7a0d-7049-4af1-adbb-533acc50621a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:35.550839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cdb301-2aac-4923-ba9c-2982372d9c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:35.551436 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb301-2aac-4923-ba9c-2982372d9c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:35.551719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb301-2aac-4923-ba9c-2982372d9c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:35.551991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cdb301-2aac-4923-ba9c-2982372d9c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:35.552719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cdb301-2aac-4923-ba9c-2982372d9c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:35.554334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cdb301-2aac-4923-ba9c-2982372d9c7d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:35.559544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:42:35.567828 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1220/2406] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:42:35.569102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b963d-6adf-40c7-8d85-4ba319e7c151 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:35.579779 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b963d-6adf-40c7-8d85-4ba319e7c151 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:35.584339 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96918f-eee2-4411-a7b3-9bf8c01ea106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:35.585222 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96918f-eee2-4411-a7b3-9bf8c01ea106 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:35.585506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96918f-eee2-4411-a7b3-9bf8c01ea106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:35.585780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96918f-eee2-4411-a7b3-9bf8c01ea106 admin admin] PATH_INFO: `/v3/projects/8343434e69cf432a8c029e364416047f/users/5b0ddeea1bbb402590e66e5e25f3722d/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:35.587760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96918f-eee2-4411-a7b3-9bf8c01ea106 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8343434e69cf432a8c029e364416047f, user_id=5b0ddeea1bbb402590e66e5e25f3722d, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:35.590757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96918f-eee2-4411-a7b3-9bf8c01ea106 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:35.601492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1187/2407] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:42:35 2022] PUT /identity/v3/projects/8343434e69cf432a8c029e364416047f/users/5b0ddeea1bbb402590e66e5e25f3722d/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:42:40.171008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d013e-8046-454a-8378-992996bf67f3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:40.171490 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d013e-8046-454a-8378-992996bf67f3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:40.171611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d013e-8046-454a-8378-992996bf67f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:40.199807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ac1d013e-8046-454a-8378-992996bf67f3 None None] MFA Rules not processed for user `5b0ddeea1bbb402590e66e5e25f3722d`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:42:40.200972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ac1d013e-8046-454a-8378-992996bf67f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:40.298569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1221/2408] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:42:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:42:40.307949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96918f-eee2-4411-a7b3-9bf8c01ea106 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:42:40.312539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-da96918f-eee2-4411-a7b3-9bf8c01ea106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:40.393875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96918f-eee2-4411-a7b3-9bf8c01ea106 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:42:40.401768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e38d62-ca21-4411-8dbe-eea650a2fbeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:42:40.402748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38d62-ca21-4411-8dbe-eea650a2fbeb service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:40.403138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38d62-ca21-4411-8dbe-eea650a2fbeb service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:40.403682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e38d62-ca21-4411-8dbe-eea650a2fbeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:40.421379 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:42:40.478187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e38d62-ca21-4411-8dbe-eea650a2fbeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:42:40.479954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e38d62-ca21-4411-8dbe-eea650a2fbeb service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:42:40.490662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1188/2409] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:42:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:42:41.676747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0372db4-85d3-4885-91dd-a5f7f33701d3 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:41.677547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0372db4-85d3-4885-91dd-a5f7f33701d3 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:41.677547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0372db4-85d3-4885-91dd-a5f7f33701d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:41.708997 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a0372db4-85d3-4885-91dd-a5f7f33701d3 None None] MFA Rules not processed for user `215c8b04d6294311aeb1124b9139c7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:42:41.710635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a0372db4-85d3-4885-91dd-a5f7f33701d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:41.722758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1222/2410] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:42:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:42:43.940081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3220aa8-ba28-45aa-9294-b9dccc68f402 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:42:43.940630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3220aa8-ba28-45aa-9294-b9dccc68f402 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:42:43.940630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3220aa8-ba28-45aa-9294-b9dccc68f402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:42:43.961548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f3220aa8-ba28-45aa-9294-b9dccc68f402 None None] MFA Rules not processed for user `ac984d1f6efa403e85483bd7fa78000e`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:42:43.962805 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f3220aa8-ba28-45aa-9294-b9dccc68f402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:42:44.022598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1189/2411] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:42:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:42:51.694200 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=37 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:42:54.015352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=48 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:43:29.446560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0372db4-85d3-4885-91dd-a5f7f33701d3 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:43:29.503462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0372db4-85d3-4885-91dd-a5f7f33701d3 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:43:29.507106 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed5725-3212-401c-b2ec-4bc03db7f180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:43:29.507759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed5725-3212-401c-b2ec-4bc03db7f180 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:43:29.508039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed5725-3212-401c-b2ec-4bc03db7f180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:43:29.508313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed5725-3212-401c-b2ec-4bc03db7f180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:43:29.522209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed5725-3212-401c-b2ec-4bc03db7f180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:43:29.524359 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed5725-3212-401c-b2ec-4bc03db7f180 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:43:29.536218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1223/2412] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:43:29 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:43:39.530738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=26 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:43:51.214592 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3220aa8-ba28-45aa-9294-b9dccc68f402 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:43:51.216639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f3220aa8-ba28-45aa-9294-b9dccc68f402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:43:51.303259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3220aa8-ba28-45aa-9294-b9dccc68f402 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:43:51.311109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00732d1b-1675-4f76-8197-093cd0e6a4f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:43:51.311109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00732d1b-1675-4f76-8197-093cd0e6a4f8 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:43:51.311109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00732d1b-1675-4f76-8197-093cd0e6a4f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:43:51.311109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00732d1b-1675-4f76-8197-093cd0e6a4f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:43:51.312262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-00732d1b-1675-4f76-8197-093cd0e6a4f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:43:51.326638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00732d1b-1675-4f76-8197-093cd0e6a4f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:43:51.328261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00732d1b-1675-4f76-8197-093cd0e6a4f8 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:43:51.341996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1190/2413] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:43:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:44:01.334757 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=32 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:44:01.774149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52769294-d24d-4bfe-af67-46fe2071a625 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:01.775237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52769294-d24d-4bfe-af67-46fe2071a625 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:01.775533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52769294-d24d-4bfe-af67-46fe2071a625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:01.795308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-52769294-d24d-4bfe-af67-46fe2071a625 None None] MFA Rules not processed for user `bccf002d9da34d1dad0ee71b5b7bb1c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:44:01.796503 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-52769294-d24d-4bfe-af67-46fe2071a625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:01.859510 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1224/2414] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:44:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:44:11.846381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=25 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:44:19.385666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f3c3-0772-4fe4-97f7-d87c9fdd8519 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:19.386215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f3c3-0772-4fe4-97f7-d87c9fdd8519 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:19.386215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f3c3-0772-4fe4-97f7-d87c9fdd8519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:19.406448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b108f3c3-0772-4fe4-97f7-d87c9fdd8519 None None] MFA Rules not processed for user `1b2337595cbb40608d6bc485889d8bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:44:19.407825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b108f3c3-0772-4fe4-97f7-d87c9fdd8519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:19.479656 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1191/2415] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:44:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:44:21.032592 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ca15e-491b-4b88-b24b-56bcd71ed91c None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:21.033076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ca15e-491b-4b88-b24b-56bcd71ed91c None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:21.033166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ca15e-491b-4b88-b24b-56bcd71ed91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:21.052818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-d31ca15e-491b-4b88-b24b-56bcd71ed91c None None] MFA Rules not processed for user `37d6c5d1f76445499566472213f13e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:44:21.054665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d31ca15e-491b-4b88-b24b-56bcd71ed91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:21.113546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1225/2416] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:44:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:44:29.287568 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108f3c3-0772-4fe4-97f7-d87c9fdd8519 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:29.299202 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108f3c3-0772-4fe4-97f7-d87c9fdd8519 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:29.303706 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcc812-2f2b-45a0-9790-2b38b5199bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:29.304355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcc812-2f2b-45a0-9790-2b38b5199bfd service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:29.304636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcc812-2f2b-45a0-9790-2b38b5199bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:29.304915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcc812-2f2b-45a0-9790-2b38b5199bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:29.307259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-76fcc812-2f2b-45a0-9790-2b38b5199bfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:29.321349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fcc812-2f2b-45a0-9790-2b38b5199bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:29.322984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fcc812-2f2b-45a0-9790-2b38b5199bfd service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:29.333174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1192/2417] 10.222.0.27 () {60 vars in 1396 bytes} [Wed Jun 29 08:44:29 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3246 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:44:31.105416 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=25 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:44:39.328762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=31 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:44:45.389133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ca15e-491b-4b88-b24b-56bcd71ed91c None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:45.399617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ca15e-491b-4b88-b24b-56bcd71ed91c None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:45.404201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb403c0-5970-4f7d-b4fe-dcb1cf627703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:45.404848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb403c0-5970-4f7d-b4fe-dcb1cf627703 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:45.405124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb403c0-5970-4f7d-b4fe-dcb1cf627703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:45.405395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb403c0-5970-4f7d-b4fe-dcb1cf627703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:45.407479 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ffb403c0-5970-4f7d-b4fe-dcb1cf627703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:45.481094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb403c0-5970-4f7d-b4fe-dcb1cf627703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:45.482540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb403c0-5970-4f7d-b4fe-dcb1cf627703 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:45.492752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1226/2418] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:44:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:44:48.685691 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcc812-2f2b-45a0-9790-2b38b5199bfd service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:48.697573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcc812-2f2b-45a0-9790-2b38b5199bfd service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:48.703456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922dbf1-4e50-46c7-8937-46a877036be6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:48.704228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922dbf1-4e50-46c7-8937-46a877036be6 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:48.704401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922dbf1-4e50-46c7-8937-46a877036be6 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:48.704806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922dbf1-4e50-46c7-8937-46a877036be6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:48.716617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6922dbf1-4e50-46c7-8937-46a877036be6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:48.729908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922dbf1-4e50-46c7-8937-46a877036be6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:48.731694 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922dbf1-4e50-46c7-8937-46a877036be6 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:48.743114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1193/2419] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:44:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:44:48.758470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb403c0-5970-4f7d-b4fe-dcb1cf627703 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:48.761148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ffb403c0-5970-4f7d-b4fe-dcb1cf627703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:48.773571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb403c0-5970-4f7d-b4fe-dcb1cf627703 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:48.778804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b9115-3aef-4863-9626-52d2626bcc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:48.779716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b9115-3aef-4863-9626-52d2626bcc54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:48.780073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b9115-3aef-4863-9626-52d2626bcc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:48.780363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b9115-3aef-4863-9626-52d2626bcc54 admin admin] PATH_INFO: `/v3/users/ac984d1f6efa403e85483bd7fa78000e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:48.781647 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b9115-3aef-4863-9626-52d2626bcc54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac984d1f6efa403e85483bd7fa78000e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:48.783661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b9115-3aef-4863-9626-52d2626bcc54 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:48.832826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1f5b9115-3aef-4863-9626-52d2626bcc54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac984d1f6efa403e85483bd7fa78000e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:44:48.833475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1f5b9115-3aef-4863-9626-52d2626bcc54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac984d1f6efa403e85483bd7fa78000e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:44:48.845942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1f5b9115-3aef-4863-9626-52d2626bcc54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac984d1f6efa403e85483bd7fa78000e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:44:48.846588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1f5b9115-3aef-4863-9626-52d2626bcc54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac984d1f6efa403e85483bd7fa78000e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:44:48.855257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-1f5b9115-3aef-4863-9626-52d2626bcc54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac984d1f6efa403e85483bd7fa78000e'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:44:48.868137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1227/2420] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:44:48 2022] DELETE /identity/v3/users/ac984d1f6efa403e85483bd7fa78000e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:44:49.066156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922dbf1-4e50-46c7-8937-46a877036be6 service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:49.078995 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922dbf1-4e50-46c7-8937-46a877036be6 service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:49.145555 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:49.146450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:49.147058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:49.147356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] PATH_INFO: `/v3/projects/96872c14d1224c92b3247285f03bb966` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:49.149329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] RBAC: Authorizing `identity:delete_project(project_id=96872c14d1224c92b3247285f03bb966)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:49.152216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:49.205588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96872c14d1224c92b3247285f03bb966', 'request_id': 'req-7d1841f8-52a5-4725-9729-570ee14d892e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:44:49.212335 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96872c14d1224c92b3247285f03bb966', 'request_id': 'req-7d1841f8-52a5-4725-9729-570ee14d892e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:44:49.219149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96872c14d1224c92b3247285f03bb966', 'request_id': 'req-7d1841f8-52a5-4725-9729-570ee14d892e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:44:49.232859 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] The token cache is being invalidate because project 96872c14d1224c92b3247285f03bb966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:44:49.233105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:44:49.233720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7d1841f8-52a5-4725-9729-570ee14d892e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:44:49.235334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1194/2421] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:44:49 2022] DELETE /identity/v3/projects/96872c14d1224c92b3247285f03bb966 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:44:49.240424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac50d84-a391-4c53-9bec-2f18b7366802 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:49.240719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac50d84-a391-4c53-9bec-2f18b7366802 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:49.241033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac50d84-a391-4c53-9bec-2f18b7366802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:49.260112 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2ac50d84-a391-4c53-9bec-2f18b7366802 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:44:49.261328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2ac50d84-a391-4c53-9bec-2f18b7366802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:49.324804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1228/2422] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:44:49.333312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a403f3-2994-4f65-8b39-09449b3fe95a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:49.333608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a403f3-2994-4f65-8b39-09449b3fe95a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:49.333887 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a403f3-2994-4f65-8b39-09449b3fe95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:49.353990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-92a403f3-2994-4f65-8b39-09449b3fe95a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:44:49.355242 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-92a403f3-2994-4f65-8b39-09449b3fe95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:49.364860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1195/2423] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:44:49.369281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac50d84-a391-4c53-9bec-2f18b7366802 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:49.379339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac50d84-a391-4c53-9bec-2f18b7366802 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:49.385479 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988033ef-b584-49f3-ab67-b83c3d309820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:49.386203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988033ef-b584-49f3-ab67-b83c3d309820 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:49.386682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988033ef-b584-49f3-ab67-b83c3d309820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:49.387152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988033ef-b584-49f3-ab67-b83c3d309820 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:49.388302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988033ef-b584-49f3-ab67-b83c3d309820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:44:49.388784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988033ef-b584-49f3-ab67-b83c3d309820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:49.391322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988033ef-b584-49f3-ab67-b83c3d309820 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:49.393426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:44:49.408171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1229/2424] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:44:49.412428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a403f3-2994-4f65-8b39-09449b3fe95a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:49.421892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a403f3-2994-4f65-8b39-09449b3fe95a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:49.427234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235ea95-7e69-4c3d-a92b-c85de01282db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:49.427884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235ea95-7e69-4c3d-a92b-c85de01282db admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:49.428165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235ea95-7e69-4c3d-a92b-c85de01282db admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:49.428436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235ea95-7e69-4c3d-a92b-c85de01282db admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:49.429124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235ea95-7e69-4c3d-a92b-c85de01282db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:49.431080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235ea95-7e69-4c3d-a92b-c85de01282db admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:49.461781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1196/2425] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:44:49 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:44:49.466536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988033ef-b584-49f3-ab67-b83c3d309820 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:49.476874 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988033ef-b584-49f3-ab67-b83c3d309820 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:49.536393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b3d65-75d7-48e3-9331-4e79d9365576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:49.536989 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b3d65-75d7-48e3-9331-4e79d9365576 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:49.537267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b3d65-75d7-48e3-9331-4e79d9365576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:49.537545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b3d65-75d7-48e3-9331-4e79d9365576 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:49.538284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b3d65-75d7-48e3-9331-4e79d9365576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:49.540274 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41b3d65-75d7-48e3-9331-4e79d9365576 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:49.562147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1230/2426] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:44:49 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:44:49.568076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235ea95-7e69-4c3d-a92b-c85de01282db admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:49.578975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235ea95-7e69-4c3d-a92b-c85de01282db admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:49.584536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f392a-a010-455e-8b78-d771d6a2724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:49.585200 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f392a-a010-455e-8b78-d771d6a2724d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:49.585483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f392a-a010-455e-8b78-d771d6a2724d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:49.585760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f392a-a010-455e-8b78-d771d6a2724d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:49.586495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f392a-a010-455e-8b78-d771d6a2724d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:49.588345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f392a-a010-455e-8b78-d771d6a2724d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:49.599271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1197/2427] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:44:49.603173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b3d65-75d7-48e3-9331-4e79d9365576 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:49.613405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b3d65-75d7-48e3-9331-4e79d9365576 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:49.618048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbe9e4-c30e-4ade-b767-7e5c9a6b7159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:49.618798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbe9e4-c30e-4ade-b767-7e5c9a6b7159 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:49.619119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbe9e4-c30e-4ade-b767-7e5c9a6b7159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:49.619394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbe9e4-c30e-4ade-b767-7e5c9a6b7159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:49.620162 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbe9e4-c30e-4ade-b767-7e5c9a6b7159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:49.621767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbe9e4-c30e-4ade-b767-7e5c9a6b7159 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:49.631811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1231/2428] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:44:49.635977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f392a-a010-455e-8b78-d771d6a2724d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:49.646670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f392a-a010-455e-8b78-d771d6a2724d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:49.652846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a08afd-c4e8-42b6-b858-755fdca23f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:49.653762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a08afd-c4e8-42b6-b858-755fdca23f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:49.654057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a08afd-c4e8-42b6-b858-755fdca23f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:49.654332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a08afd-c4e8-42b6-b858-755fdca23f92 admin admin] PATH_INFO: `/v3/projects/7faee22e84ba4b8b85f4d4cf6132d83f/users/88a1f82bc0924a84849d390f138041a1/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:49.670952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a08afd-c4e8-42b6-b858-755fdca23f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7faee22e84ba4b8b85f4d4cf6132d83f, user_id=88a1f82bc0924a84849d390f138041a1, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:49.673608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a08afd-c4e8-42b6-b858-755fdca23f92 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:49.687887 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1198/2429] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:44:49 2022] PUT /identity/v3/projects/7faee22e84ba4b8b85f4d4cf6132d83f/users/88a1f82bc0924a84849d390f138041a1/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:44:49.693219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8c17b-5639-4a38-8404-61b3d87997ee None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:49.693511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8c17b-5639-4a38-8404-61b3d87997ee None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:49.693788 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8c17b-5639-4a38-8404-61b3d87997ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:49.711909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2be8c17b-5639-4a38-8404-61b3d87997ee None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:44:49.713144 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2be8c17b-5639-4a38-8404-61b3d87997ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:49.773416 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1232/2430] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:44:49.783478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a08afd-c4e8-42b6-b858-755fdca23f92 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:49.785577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-47a08afd-c4e8-42b6-b858-755fdca23f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:49.801770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:44:49.848387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a08afd-c4e8-42b6-b858-755fdca23f92 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:49.852164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a7eb1-9536-4ca7-85fc-86919828e94f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:49.852890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7eb1-9536-4ca7-85fc-86919828e94f service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:49.853175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7eb1-9536-4ca7-85fc-86919828e94f service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:49.853509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a7eb1-9536-4ca7-85fc-86919828e94f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:49.862911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a7eb1-9536-4ca7-85fc-86919828e94f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:49.864347 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a7eb1-9536-4ca7-85fc-86919828e94f service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:49.874169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1199/2431] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:44:53.751793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce6636-72e2-4e89-ad04-57c11dc14b9b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:53.752214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce6636-72e2-4e89-ad04-57c11dc14b9b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:53.752214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce6636-72e2-4e89-ad04-57c11dc14b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:53.785450 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a9ce6636-72e2-4e89-ad04-57c11dc14b9b None None] MFA Rules not processed for user `88a1f82bc0924a84849d390f138041a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:44:53.786851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a9ce6636-72e2-4e89-ad04-57c11dc14b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:53.864062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1233/2432] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:44:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:44:53.881013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a7eb1-9536-4ca7-85fc-86919828e94f service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:44:53.882610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b37a7eb1-9536-4ca7-85fc-86919828e94f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:53.975603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a7eb1-9536-4ca7-85fc-86919828e94f service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:44:53.981114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678812e5-336c-4874-9397-66d147d47342 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:44:53.981774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678812e5-336c-4874-9397-66d147d47342 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:53.982054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678812e5-336c-4874-9397-66d147d47342 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:53.982332 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678812e5-336c-4874-9397-66d147d47342 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:53.991223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678812e5-336c-4874-9397-66d147d47342 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:44:53.992842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678812e5-336c-4874-9397-66d147d47342 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:44:54.002809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1200/2433] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:44:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:44:55.375733 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea370c7-aa7f-4c4b-a404-8e1b4ae4839f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:44:55.376245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea370c7-aa7f-4c4b-a404-8e1b4ae4839f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:44:55.376341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea370c7-aa7f-4c4b-a404-8e1b4ae4839f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:44:55.382276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:44:55.397065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5ea370c7-aa7f-4c4b-a404-8e1b4ae4839f None None] MFA Rules not processed for user `88a1f82bc0924a84849d390f138041a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:44:55.398285 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5ea370c7-aa7f-4c4b-a404-8e1b4ae4839f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:44:55.406419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1234/2434] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:44:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:45:03.998766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=54 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:45:05.392693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:46:02.017146 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678812e5-336c-4874-9397-66d147d47342 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:02.019063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-678812e5-336c-4874-9397-66d147d47342 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:02.035301 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678812e5-336c-4874-9397-66d147d47342 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:02.041450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b46b8-fdd4-4602-9121-1d2d35e41399 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:02.042087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b46b8-fdd4-4602-9121-1d2d35e41399 service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:02.042373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b46b8-fdd4-4602-9121-1d2d35e41399 service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:02.042640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b46b8-fdd4-4602-9121-1d2d35e41399 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:02.044485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-503b46b8-fdd4-4602-9121-1d2d35e41399 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:02.128153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b46b8-fdd4-4602-9121-1d2d35e41399 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:02.129743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b46b8-fdd4-4602-9121-1d2d35e41399 service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:02.149372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1201/2435] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:46:04.630757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea370c7-aa7f-4c4b-a404-8e1b4ae4839f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:04.632363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5ea370c7-aa7f-4c4b-a404-8e1b4ae4839f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:04.692923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea370c7-aa7f-4c4b-a404-8e1b4ae4839f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:04.697223 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869061e-7946-4809-8a90-6e1818797dad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:04.697869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869061e-7946-4809-8a90-6e1818797dad service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:04.698150 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869061e-7946-4809-8a90-6e1818797dad service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:04.698431 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869061e-7946-4809-8a90-6e1818797dad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:04.700183 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7869061e-7946-4809-8a90-6e1818797dad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:04.720563 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869061e-7946-4809-8a90-6e1818797dad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:04.721921 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869061e-7946-4809-8a90-6e1818797dad service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:04.733050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1235/2436] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:10.454288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b46b8-fdd4-4602-9121-1d2d35e41399 service nova] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:10.464569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b46b8-fdd4-4602-9121-1d2d35e41399 service nova] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:10.469034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d91d69-8b4d-4734-9dde-1415b7279919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:10.469683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d91d69-8b4d-4734-9dde-1415b7279919 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:10.469973 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d91d69-8b4d-4734-9dde-1415b7279919 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:10.470244 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d91d69-8b4d-4734-9dde-1415b7279919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:10.472206 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-01d91d69-8b4d-4734-9dde-1415b7279919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:10.487776 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d91d69-8b4d-4734-9dde-1415b7279919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:10.489224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d91d69-8b4d-4734-9dde-1415b7279919 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:10.500821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1202/2437] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:14.728475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=26 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:46:17.226887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869061e-7946-4809-8a90-6e1818797dad service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:17.239585 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869061e-7946-4809-8a90-6e1818797dad service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:17.245530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42b304-8317-4eb6-b1a7-1bcc30e8ff37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:17.246184 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42b304-8317-4eb6-b1a7-1bcc30e8ff37 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:17.246540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42b304-8317-4eb6-b1a7-1bcc30e8ff37 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:17.246869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42b304-8317-4eb6-b1a7-1bcc30e8ff37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:17.249389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fb42b304-8317-4eb6-b1a7-1bcc30e8ff37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:17.269475 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42b304-8317-4eb6-b1a7-1bcc30e8ff37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:17.272130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42b304-8317-4eb6-b1a7-1bcc30e8ff37 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:17.291731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1236/2438] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:19.507470 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d91d69-8b4d-4734-9dde-1415b7279919 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:19.508975 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-01d91d69-8b4d-4734-9dde-1415b7279919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:19.521023 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d91d69-8b4d-4734-9dde-1415b7279919 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:19.525054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e0763-5120-47d7-a58a-480c74feae21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:19.525704 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e0763-5120-47d7-a58a-480c74feae21 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:19.525986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e0763-5120-47d7-a58a-480c74feae21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:19.526254 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e0763-5120-47d7-a58a-480c74feae21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:19.528153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-125e0763-5120-47d7-a58a-480c74feae21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:19.538084 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e0763-5120-47d7-a58a-480c74feae21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:19.539548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e0763-5120-47d7-a58a-480c74feae21 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:19.548522 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1203/2439] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:20.752637 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42b304-8317-4eb6-b1a7-1bcc30e8ff37 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:20.764840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42b304-8317-4eb6-b1a7-1bcc30e8ff37 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:20.770887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b973a3e-09ee-4dcf-97b8-5f5c0e02cabc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:20.771544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973a3e-09ee-4dcf-97b8-5f5c0e02cabc service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:20.771830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973a3e-09ee-4dcf-97b8-5f5c0e02cabc service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:20.772109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973a3e-09ee-4dcf-97b8-5f5c0e02cabc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:20.774018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2b973a3e-09ee-4dcf-97b8-5f5c0e02cabc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:20.858777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b973a3e-09ee-4dcf-97b8-5f5c0e02cabc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:20.860456 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b973a3e-09ee-4dcf-97b8-5f5c0e02cabc service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:20.879307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1237/2440] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:21.370791 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989382f1-e661-4805-b16a-6142d80542fe None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:21.371574 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989382f1-e661-4805-b16a-6142d80542fe None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:21.372154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989382f1-e661-4805-b16a-6142d80542fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:21.403154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-989382f1-e661-4805-b16a-6142d80542fe None None] MFA Rules not processed for user `bccf002d9da34d1dad0ee71b5b7bb1c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:46:21.404377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-989382f1-e661-4805-b16a-6142d80542fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:21.411900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1204/2441] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:46:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:46:21.757025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b973a3e-09ee-4dcf-97b8-5f5c0e02cabc service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:21.769071 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b973a3e-09ee-4dcf-97b8-5f5c0e02cabc service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:21.773473 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71604a79-41e3-4cfd-85dd-8387afb3e9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:21.774157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71604a79-41e3-4cfd-85dd-8387afb3e9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:21.774439 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71604a79-41e3-4cfd-85dd-8387afb3e9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:21.774734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71604a79-41e3-4cfd-85dd-8387afb3e9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:21.776519 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-71604a79-41e3-4cfd-85dd-8387afb3e9d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:21.788019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71604a79-41e3-4cfd-85dd-8387afb3e9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:21.790098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71604a79-41e3-4cfd-85dd-8387afb3e9d3 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:21.800679 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1238/2442] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:22.131452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989382f1-e661-4805-b16a-6142d80542fe None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:22.132861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-989382f1-e661-4805-b16a-6142d80542fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:22.210514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989382f1-e661-4805-b16a-6142d80542fe None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:22.215889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6ca40-c81d-4611-9021-bc250399bc1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:22.216562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6ca40-c81d-4611-9021-bc250399bc1c service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:22.216847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6ca40-c81d-4611-9021-bc250399bc1c service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:22.217151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6ca40-c81d-4611-9021-bc250399bc1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:22.219028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a8f6ca40-c81d-4611-9021-bc250399bc1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:22.231099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6ca40-c81d-4611-9021-bc250399bc1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:22.233117 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6ca40-c81d-4611-9021-bc250399bc1c service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:22.253844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1205/2443] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:22.799966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71604a79-41e3-4cfd-85dd-8387afb3e9d3 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:22.809707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71604a79-41e3-4cfd-85dd-8387afb3e9d3 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:22.814922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f04ba-3fa1-47fb-8370-2b47a31cffa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:22.815580 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f04ba-3fa1-47fb-8370-2b47a31cffa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:22.815862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f04ba-3fa1-47fb-8370-2b47a31cffa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:22.816136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f04ba-3fa1-47fb-8370-2b47a31cffa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:22.818257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-731f04ba-3fa1-47fb-8370-2b47a31cffa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:22.829855 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f04ba-3fa1-47fb-8370-2b47a31cffa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:22.831397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f04ba-3fa1-47fb-8370-2b47a31cffa4 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:22.839870 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1239/2444] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:22.950909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dbe29-4aaf-4c48-ae34-eebdb62bda32 None None] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:22.951952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dbe29-4aaf-4c48-ae34-eebdb62bda32 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:22.952463 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357dbe29-4aaf-4c48-ae34-eebdb62bda32 None None] PATH_INFO: `/` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:22.954214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1206/2445] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:46:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:46:22.961072 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4fae2-a9c2-4cbc-af76-80b0b3a61b93 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:22.961388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4fae2-a9c2-4cbc-af76-80b0b3a61b93 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:22.961668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4fae2-a9c2-4cbc-af76-80b0b3a61b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:23.005138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-37c4fae2-a9c2-4cbc-af76-80b0b3a61b93 None None] MFA Rules not processed for user `49a1439029fa4b719bc114c3f59e0050`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:46:23.006606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-37c4fae2-a9c2-4cbc-af76-80b0b3a61b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:23.014303 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1240/2446] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:46:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:46:23.026666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dbe29-4aaf-4c48-ae34-eebdb62bda32 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:23.037799 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357dbe29-4aaf-4c48-ae34-eebdb62bda32 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:23.041355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100af5a-60da-4312-8209-bb5a916c8ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:23.042002 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100af5a-60da-4312-8209-bb5a916c8ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:23.042283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100af5a-60da-4312-8209-bb5a916c8ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:23.042553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100af5a-60da-4312-8209-bb5a916c8ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:23.050796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100af5a-60da-4312-8209-bb5a916c8ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:23.052594 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100af5a-60da-4312-8209-bb5a916c8ee5 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:23.061113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1207/2447] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:23.151404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5c892-0ec8-423a-80a6-c777e5366851 None None] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:23.151743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5c892-0ec8-423a-80a6-c777e5366851 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:23.152034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5c892-0ec8-423a-80a6-c777e5366851 None None] PATH_INFO: `/` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:23.153512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1241/2448] 10.222.0.27 () {54 vars in 921 bytes} [Wed Jun 29 08:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 08:46:23.160568 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afa637-6421-4f28-b10b-279f88f5f073 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:23.160853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afa637-6421-4f28-b10b-279f88f5f073 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:23.161311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30afa637-6421-4f28-b10b-279f88f5f073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:23.175405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:46:23.181854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-30afa637-6421-4f28-b10b-279f88f5f073 None None] MFA Rules not processed for user `7bd500f020f143339c5a4fc9f0728ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:46:23.183093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-30afa637-6421-4f28-b10b-279f88f5f073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:23.193851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1208/2449] 10.222.0.27 () {58 vars in 1018 bytes} [Wed Jun 29 08:46:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:46:23.207876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5c892-0ec8-423a-80a6-c777e5366851 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:23.222152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5c892-0ec8-423a-80a6-c777e5366851 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:23.226344 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4635-8c0c-42b2-8cf7-a2d291eb68c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:23.227492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4635-8c0c-42b2-8cf7-a2d291eb68c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:23.227783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4635-8c0c-42b2-8cf7-a2d291eb68c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:23.228067 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e4635-8c0c-42b2-8cf7-a2d291eb68c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:23.243827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e4635-8c0c-42b2-8cf7-a2d291eb68c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:23.245690 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e4635-8c0c-42b2-8cf7-a2d291eb68c6 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:23.275388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1242/2450] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:24.806416 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afa637-6421-4f28-b10b-279f88f5f073 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:24.819051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30afa637-6421-4f28-b10b-279f88f5f073 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:24.824991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447679d8-3928-4fe9-8a82-c6aae4cf3a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:24.825648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447679d8-3928-4fe9-8a82-c6aae4cf3a7a service nova] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:24.825926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447679d8-3928-4fe9-8a82-c6aae4cf3a7a service nova] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:24.826221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447679d8-3928-4fe9-8a82-c6aae4cf3a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:24.828079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-447679d8-3928-4fe9-8a82-c6aae4cf3a7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:24.841623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447679d8-3928-4fe9-8a82-c6aae4cf3a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:24.843328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447679d8-3928-4fe9-8a82-c6aae4cf3a7a service nova] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:24.855575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1209/2451] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:27.691210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4635-8c0c-42b2-8cf7-a2d291eb68c6 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:27.705954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e4635-8c0c-42b2-8cf7-a2d291eb68c6 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:27.713658 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88d40f-476a-46b7-be86-d4f540d76931 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:27.714402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88d40f-476a-46b7-be86-d4f540d76931 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:27.714940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88d40f-476a-46b7-be86-d4f540d76931 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:27.715701 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88d40f-476a-46b7-be86-d4f540d76931 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:27.719032 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8b88d40f-476a-46b7-be86-d4f540d76931 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:27.733709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88d40f-476a-46b7-be86-d4f540d76931 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:27.735763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88d40f-476a-46b7-be86-d4f540d76931 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:27.765505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1243/2452] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:46:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:46:34.850367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:46:37.742752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=80 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:46:46.216702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bacf4-c3f5-4cfe-9572-72a850814bf5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:46.217223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bacf4-c3f5-4cfe-9572-72a850814bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:46.217223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bacf4-c3f5-4cfe-9572-72a850814bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:46.234998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c24bacf4-c3f5-4cfe-9572-72a850814bf5 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:46:46.236200 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c24bacf4-c3f5-4cfe-9572-72a850814bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:46.290997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1210/2453] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:46:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:46:46.342884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88d40f-476a-46b7-be86-d4f540d76931 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:46:46.344334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8b88d40f-476a-46b7-be86-d4f540d76931 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:46:46.418810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88d40f-476a-46b7-be86-d4f540d76931 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:46:46.424831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd93c4-2606-463d-a592-898db64218ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:46:46.425479 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd93c4-2606-463d-a592-898db64218ea service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:46:46.425765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd93c4-2606-463d-a592-898db64218ea service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:46:46.426037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd93c4-2606-463d-a592-898db64218ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:46:46.435981 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd93c4-2606-463d-a592-898db64218ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:46:46.438337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fd93c4-2606-463d-a592-898db64218ea service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:46:46.448942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1244/2454] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:46:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:46:56.284027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=25 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:46:56.445755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=32 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:47:04.866857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bacf4-c3f5-4cfe-9572-72a850814bf5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:47:04.868381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c24bacf4-c3f5-4cfe-9572-72a850814bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:47:04.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bacf4-c3f5-4cfe-9572-72a850814bf5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:47:04.932884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ea69c-847d-4705-bfb7-2b59cc227635 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd09453b5db645d7b03522f91effb8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:47:04.933548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ea69c-847d-4705-bfb7-2b59cc227635 service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:47:04.933822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ea69c-847d-4705-bfb7-2b59cc227635 service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:47:04.934100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ea69c-847d-4705-bfb7-2b59cc227635 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:47:04.935917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a11ea69c-847d-4705-bfb7-2b59cc227635 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:47:04.950051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ea69c-847d-4705-bfb7-2b59cc227635 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:47:04.951525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ea69c-847d-4705-bfb7-2b59cc227635 service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:47:04.964534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1211/2455] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:47:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:47:05.041544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd93c4-2606-463d-a592-898db64218ea service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:47:05.052340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd93c4-2606-463d-a592-898db64218ea service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:47:05.056088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89eac73-2571-4f9d-9876-9505318238c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:47:05.056736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eac73-2571-4f9d-9876-9505318238c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:47:05.057019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eac73-2571-4f9d-9876-9505318238c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:47:05.057294 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eac73-2571-4f9d-9876-9505318238c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:47:05.059211 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b89eac73-2571-4f9d-9876-9505318238c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:47:05.068968 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89eac73-2571-4f9d-9876-9505318238c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:47:05.070400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89eac73-2571-4f9d-9876-9505318238c3 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:47:05.079024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1245/2456] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:47:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:47:05.153864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ea69c-847d-4705-bfb7-2b59cc227635 service glance] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:47:05.168490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ea69c-847d-4705-bfb7-2b59cc227635 service glance] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:47:05.172774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15613693-48ff-4036-a050-96d49e5da84b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:47:05.173422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15613693-48ff-4036-a050-96d49e5da84b service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:47:05.173709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15613693-48ff-4036-a050-96d49e5da84b service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:47:05.173991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15613693-48ff-4036-a050-96d49e5da84b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:47:05.176099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-15613693-48ff-4036-a050-96d49e5da84b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:47:05.190703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15613693-48ff-4036-a050-96d49e5da84b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:47:05.192154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15613693-48ff-4036-a050-96d49e5da84b service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:47:05.203665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1212/2457] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:47:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:47:07.105729 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b8ae8-99a0-46d0-a0d6-0f416bdbd00a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:47:07.106235 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b8ae8-99a0-46d0-a0d6-0f416bdbd00a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:47:07.106235 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b8ae8-99a0-46d0-a0d6-0f416bdbd00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:47:07.126059 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-398b8ae8-99a0-46d0-a0d6-0f416bdbd00a None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:47:07.127294 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-398b8ae8-99a0-46d0-a0d6-0f416bdbd00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:47:07.136246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1246/2458] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:47:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:47:07.151496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15613693-48ff-4036-a050-96d49e5da84b service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:47:07.169110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15613693-48ff-4036-a050-96d49e5da84b service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:47:07.175246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6398334-3daa-4a0b-9356-fbbba0c284a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:47:07.175883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6398334-3daa-4a0b-9356-fbbba0c284a8 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:47:07.176166 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6398334-3daa-4a0b-9356-fbbba0c284a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:47:07.176428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6398334-3daa-4a0b-9356-fbbba0c284a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:47:07.184623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6398334-3daa-4a0b-9356-fbbba0c284a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:47:07.186228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6398334-3daa-4a0b-9356-fbbba0c284a8 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:47:07.194138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1213/2459] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:47:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:47:09.017440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff040d3-3bcd-4a2a-92ae-b82ef9474037 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:47:09.018073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff040d3-3bcd-4a2a-92ae-b82ef9474037 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:47:09.018073 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff040d3-3bcd-4a2a-92ae-b82ef9474037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:47:09.036028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8ff040d3-3bcd-4a2a-92ae-b82ef9474037 None None] MFA Rules not processed for user `1b2337595cbb40608d6bc485889d8bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:47:09.037260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8ff040d3-3bcd-4a2a-92ae-b82ef9474037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:47:09.099166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1247/2460] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:47:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:47:17.191096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=38 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:47:19.092300 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=34 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:47:31.357955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d217815-96d5-4799-9bb1-36d36c67c574 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:47:31.358479 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d217815-96d5-4799-9bb1-36d36c67c574 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:47:31.358583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d217815-96d5-4799-9bb1-36d36c67c574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:47:31.377618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8d217815-96d5-4799-9bb1-36d36c67c574 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:47:31.378861 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8d217815-96d5-4799-9bb1-36d36c67c574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:47:31.387442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1214/2461] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jun 29 08:47:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:47:31.400305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff040d3-3bcd-4a2a-92ae-b82ef9474037 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:47:31.412628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff040d3-3bcd-4a2a-92ae-b82ef9474037 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:47:31.418369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9661ec6-6f42-4369-8323-ca34f207e306 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:47:31.419170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9661ec6-6f42-4369-8323-ca34f207e306 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:47:31.419451 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9661ec6-6f42-4369-8323-ca34f207e306 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:47:31.419781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9661ec6-6f42-4369-8323-ca34f207e306 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:47:31.428180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9661ec6-6f42-4369-8323-ca34f207e306 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:47:31.429784 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9661ec6-6f42-4369-8323-ca34f207e306 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:47:31.437749 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1248/2462] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:47:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:47:41.371933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:47:41.434600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:47:46.484021 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d217815-96d5-4799-9bb1-36d36c67c574 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:47:46.505178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d217815-96d5-4799-9bb1-36d36c67c574 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:47:46.509618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b304379-f81c-418e-a7f4-c03d2c635fa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:47:46.510264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b304379-f81c-418e-a7f4-c03d2c635fa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:47:46.510771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b304379-f81c-418e-a7f4-c03d2c635fa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:47:46.511049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b304379-f81c-418e-a7f4-c03d2c635fa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:47:46.513071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9b304379-f81c-418e-a7f4-c03d2c635fa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:47:46.609985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b304379-f81c-418e-a7f4-c03d2c635fa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:47:46.611452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b304379-f81c-418e-a7f4-c03d2c635fa8 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:47:46.621464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1215/2463] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:47:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:47:56.617000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=48 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:48:44.137471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9661ec6-6f42-4369-8323-ca34f207e306 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:48:44.181484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9661ec6-6f42-4369-8323-ca34f207e306 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:48:44.185685 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593feacf-8c34-4253-a3c2-5c2e3bf8d918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:48:44.186322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593feacf-8c34-4253-a3c2-5c2e3bf8d918 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:48:44.186604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593feacf-8c34-4253-a3c2-5c2e3bf8d918 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:48:44.186902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593feacf-8c34-4253-a3c2-5c2e3bf8d918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:48:44.188887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-593feacf-8c34-4253-a3c2-5c2e3bf8d918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:48:44.254715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593feacf-8c34-4253-a3c2-5c2e3bf8d918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:48:44.256097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593feacf-8c34-4253-a3c2-5c2e3bf8d918 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:48:44.265207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1249/2464] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:48:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:48:51.576684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b304379-f81c-418e-a7f4-c03d2c635fa8 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:48:51.578254 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9b304379-f81c-418e-a7f4-c03d2c635fa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:48:51.618404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b304379-f81c-418e-a7f4-c03d2c635fa8 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:48:51.623556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd011e8d-cc21-443c-9334-6dacc9b59e4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:48:51.624221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011e8d-cc21-443c-9334-6dacc9b59e4c service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:48:51.624507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011e8d-cc21-443c-9334-6dacc9b59e4c service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:48:51.624789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011e8d-cc21-443c-9334-6dacc9b59e4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:48:51.626723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cd011e8d-cc21-443c-9334-6dacc9b59e4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:48:51.640671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd011e8d-cc21-443c-9334-6dacc9b59e4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:48:51.642434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd011e8d-cc21-443c-9334-6dacc9b59e4c service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:48:51.667048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1216/2465] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:48:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:48:52.703579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593feacf-8c34-4253-a3c2-5c2e3bf8d918 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:48:52.716902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593feacf-8c34-4253-a3c2-5c2e3bf8d918 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:48:52.721785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f92d8-d850-4e2c-a4c8-daf582dd9ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:48:52.723945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f92d8-d850-4e2c-a4c8-daf582dd9ffc service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:48:52.724446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f92d8-d850-4e2c-a4c8-daf582dd9ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:48:52.725608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f92d8-d850-4e2c-a4c8-daf582dd9ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:48:52.727907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e15f92d8-d850-4e2c-a4c8-daf582dd9ffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:48:52.805447 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f92d8-d850-4e2c-a4c8-daf582dd9ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:48:52.811065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f92d8-d850-4e2c-a4c8-daf582dd9ffc service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:48:52.833659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1250/2466] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:48:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:48:57.697907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd011e8d-cc21-443c-9334-6dacc9b59e4c service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:48:57.707711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd011e8d-cc21-443c-9334-6dacc9b59e4c service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:48:57.712867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d658284-dcc8-4550-a806-c74cfcb23aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:48:57.713821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d658284-dcc8-4550-a806-c74cfcb23aed service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:48:57.714633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d658284-dcc8-4550-a806-c74cfcb23aed service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:48:57.715135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d658284-dcc8-4550-a806-c74cfcb23aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:48:57.718040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7d658284-dcc8-4550-a806-c74cfcb23aed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:48:57.733959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d658284-dcc8-4550-a806-c74cfcb23aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:48:57.735777 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d658284-dcc8-4550-a806-c74cfcb23aed service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:48:57.746472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1217/2467] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:48:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:01.287631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f92d8-d850-4e2c-a4c8-daf582dd9ffc service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:01.321274 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f92d8-d850-4e2c-a4c8-daf582dd9ffc service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:01.334250 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520b267-fd54-4de0-b7e1-59f26c4e8c98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:01.335478 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520b267-fd54-4de0-b7e1-59f26c4e8c98 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:01.335701 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520b267-fd54-4de0-b7e1-59f26c4e8c98 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:01.336795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520b267-fd54-4de0-b7e1-59f26c4e8c98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:01.339394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5520b267-fd54-4de0-b7e1-59f26c4e8c98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:01.367590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520b267-fd54-4de0-b7e1-59f26c4e8c98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:01.371578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5520b267-fd54-4de0-b7e1-59f26c4e8c98 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:01.383538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d658284-dcc8-4550-a806-c74cfcb23aed service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:01.387018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7d658284-dcc8-4550-a806-c74cfcb23aed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:01.389513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1251/2468] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:01.407923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d658284-dcc8-4550-a806-c74cfcb23aed service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:01.417827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:01.418909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:01.419256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:01.419598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] PATH_INFO: `/v3/users/215c8b04d6294311aeb1124b9139c7fc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:01.420902 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=215c8b04d6294311aeb1124b9139c7fc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:01.423013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:01.493383 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215c8b04d6294311aeb1124b9139c7fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.494104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215c8b04d6294311aeb1124b9139c7fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.508595 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215c8b04d6294311aeb1124b9139c7fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.509323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215c8b04d6294311aeb1124b9139c7fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.521806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215c8b04d6294311aeb1124b9139c7fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.540658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1218/2469] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:01 2022] DELETE /identity/v3/users/215c8b04d6294311aeb1124b9139c7fc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:01.657400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520b267-fd54-4de0-b7e1-59f26c4e8c98 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:01.659181 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5520b267-fd54-4de0-b7e1-59f26c4e8c98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:01.677323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520b267-fd54-4de0-b7e1-59f26c4e8c98 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:01.740985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:01.741138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:01.742208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:01.742334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:01.742723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:01.743109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] PATH_INFO: `/v3/users/37d6c5d1f76445499566472213f13e3a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:01.744990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] RBAC: Authorizing `identity:delete_user(user_id=37d6c5d1f76445499566472213f13e3a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:01.748219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:01.756117 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbf4e5-07c9-4241-a43e-944367337997 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:01.762032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:01.763215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:01.763821 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:01.764266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] PATH_INFO: `/v3/projects/40b4ce1d85294063aea6df4b9638c90f` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:01.765727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] RBAC: Authorizing `identity:delete_project(project_id=40b4ce1d85294063aea6df4b9638c90f)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:01.767874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:01.774416 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:01.804781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37d6c5d1f76445499566472213f13e3a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.805540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37d6c5d1f76445499566472213f13e3a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.818324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37d6c5d1f76445499566472213f13e3a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.819008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37d6c5d1f76445499566472213f13e3a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.822798 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40b4ce1d85294063aea6df4b9638c90f', 'request_id': 'req-256ee075-4ee9-4b4b-b542-e5851f03f48e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.828444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37d6c5d1f76445499566472213f13e3a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.829674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40b4ce1d85294063aea6df4b9638c90f', 'request_id': 'req-256ee075-4ee9-4b4b-b542-e5851f03f48e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.835968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40b4ce1d85294063aea6df4b9638c90f', 'request_id': 'req-256ee075-4ee9-4b4b-b542-e5851f03f48e'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.842065 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1252/2470] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:01 2022] DELETE /identity/v3/users/37d6c5d1f76445499566472213f13e3a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:01.850068 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] The token cache is being invalidate because project 40b4ce1d85294063aea6df4b9638c90f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:01.850360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.851006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:01.854229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1219/2471] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:01 2022] DELETE /identity/v3/projects/40b4ce1d85294063aea6df4b9638c90f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:01.860068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:01.861355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:01.979602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdf479-5739-43e4-8475-064fc3dde66f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:01.988362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8b496-caa2-4585-b04e-7118035ef723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:01.989561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8b496-caa2-4585-b04e-7118035ef723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:01.990026 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8b496-caa2-4585-b04e-7118035ef723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:01.990366 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8b496-caa2-4585-b04e-7118035ef723 admin admin] PATH_INFO: `/v3/users/5b0ddeea1bbb402590e66e5e25f3722d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:01.992058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8b496-caa2-4585-b04e-7118035ef723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b0ddeea1bbb402590e66e5e25f3722d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:01.994353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8b496-caa2-4585-b04e-7118035ef723 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:02.066389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:02.067789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:02.074395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81b8b496-caa2-4585-b04e-7118035ef723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b0ddeea1bbb402590e66e5e25f3722d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.079959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81b8b496-caa2-4585-b04e-7118035ef723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b0ddeea1bbb402590e66e5e25f3722d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.080993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:02.104865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81b8b496-caa2-4585-b04e-7118035ef723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b0ddeea1bbb402590e66e5e25f3722d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.106482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81b8b496-caa2-4585-b04e-7118035ef723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b0ddeea1bbb402590e66e5e25f3722d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.118062 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81b8b496-caa2-4585-b04e-7118035ef723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b0ddeea1bbb402590e66e5e25f3722d'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.134545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1253/2472] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:01 2022] DELETE /identity/v3/users/5b0ddeea1bbb402590e66e5e25f3722d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:02.152818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ee075-4ee9-4b4b-b542-e5851f03f48e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:02.219992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:02.221180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:02.221574 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:02.221846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] PATH_INFO: `/v3/projects/c3addc279ffb4a49b379a938b1fbed3c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:02.223248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3addc279ffb4a49b379a938b1fbed3c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:02.225176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:02.274292 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3addc279ffb4a49b379a938b1fbed3c', 'request_id': 'req-5e50b576-312d-4acb-b002-2b1927b65c69'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.280762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3addc279ffb4a49b379a938b1fbed3c', 'request_id': 'req-5e50b576-312d-4acb-b002-2b1927b65c69'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.286810 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3addc279ffb4a49b379a938b1fbed3c', 'request_id': 'req-5e50b576-312d-4acb-b002-2b1927b65c69'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.303744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] The token cache is being invalidate because project c3addc279ffb4a49b379a938b1fbed3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:02.304039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.304693 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5e50b576-312d-4acb-b002-2b1927b65c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.306279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1220/2473] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:02 2022] DELETE /identity/v3/projects/c3addc279ffb4a49b379a938b1fbed3c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:02.311406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24371e65-1e8c-4ed6-aec8-2724a8c9df2e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:02.311937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24371e65-1e8c-4ed6-aec8-2724a8c9df2e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:02.312033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24371e65-1e8c-4ed6-aec8-2724a8c9df2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:02.332777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:02.378103 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-24371e65-1e8c-4ed6-aec8-2724a8c9df2e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:02.379292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-24371e65-1e8c-4ed6-aec8-2724a8c9df2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:02.443673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1254/2474] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:02.451927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78c828-d12b-492f-b46c-abda649ad33a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:02.452507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78c828-d12b-492f-b46c-abda649ad33a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:02.452507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78c828-d12b-492f-b46c-abda649ad33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:02.476025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9e78c828-d12b-492f-b46c-abda649ad33a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:02.477250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9e78c828-d12b-492f-b46c-abda649ad33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:02.485800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1221/2475] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:02.490824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24371e65-1e8c-4ed6-aec8-2724a8c9df2e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:02.501453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24371e65-1e8c-4ed6-aec8-2724a8c9df2e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:02.506941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e60abd-2211-45fe-bef2-9f99a0e8e3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:02.507635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e60abd-2211-45fe-bef2-9f99a0e8e3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:02.507914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e60abd-2211-45fe-bef2-9f99a0e8e3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:02.508213 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e60abd-2211-45fe-bef2-9f99a0e8e3a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:02.508955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e60abd-2211-45fe-bef2-9f99a0e8e3a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:02.509263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e60abd-2211-45fe-bef2-9f99a0e8e3a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:02.511028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e60abd-2211-45fe-bef2-9f99a0e8e3a3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:02.523795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1255/2476] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:49:02.528864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78c828-d12b-492f-b46c-abda649ad33a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:02.538641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78c828-d12b-492f-b46c-abda649ad33a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:02.543509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8d36d-b233-449e-b493-cb3068c8d020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:02.544354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8d36d-b233-449e-b493-cb3068c8d020 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:02.544649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8d36d-b233-449e-b493-cb3068c8d020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:02.544952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8d36d-b233-449e-b493-cb3068c8d020 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:02.545797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8d36d-b233-449e-b493-cb3068c8d020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:02.548269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d8d36d-b233-449e-b493-cb3068c8d020 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:02.573220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e60abd-2211-45fe-bef2-9f99a0e8e3a3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:02.575551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-32e60abd-2211-45fe-bef2-9f99a0e8e3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:02.581584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1222/2477] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:02 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:02.585918 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8d36d-b233-449e-b493-cb3068c8d020 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:02.597056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e60abd-2211-45fe-bef2-9f99a0e8e3a3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:02.609102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:02.657121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8d36d-b233-449e-b493-cb3068c8d020 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:02.659444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:02.660347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:02.660622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:02.660891 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] PATH_INFO: `/v3/projects/8343434e69cf432a8c029e364416047f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:02.661658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8b234-9c2a-4c32-bad1-6c5d32e7680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:02.662075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8343434e69cf432a8c029e364416047f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:02.662336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8b234-9c2a-4c32-bad1-6c5d32e7680d admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:02.662621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8b234-9c2a-4c32-bad1-6c5d32e7680d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:02.662927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8b234-9c2a-4c32-bad1-6c5d32e7680d admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:02.663630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8b234-9c2a-4c32-bad1-6c5d32e7680d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:02.664042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:02.665543 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8b234-9c2a-4c32-bad1-6c5d32e7680d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:02.686212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1223/2478] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:02 2022] POST /identity/v3/users => generated 303 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:02.690405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8b234-9c2a-4c32-bad1-6c5d32e7680d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:02.701241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8b234-9c2a-4c32-bad1-6c5d32e7680d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:02.706249 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba33a56-e7cd-4678-b032-192f1eda81dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:02.706960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba33a56-e7cd-4678-b032-192f1eda81dd admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:02.707240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba33a56-e7cd-4678-b032-192f1eda81dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:02.707514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba33a56-e7cd-4678-b032-192f1eda81dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:02.708248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba33a56-e7cd-4678-b032-192f1eda81dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:02.709849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba33a56-e7cd-4678-b032-192f1eda81dd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:02.717960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8343434e69cf432a8c029e364416047f', 'request_id': 'req-97507ceb-49f8-4473-8779-67a604e8185a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.720194 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1224/2479] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:02.724123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba33a56-e7cd-4678-b032-192f1eda81dd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:02.724594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8343434e69cf432a8c029e364416047f', 'request_id': 'req-97507ceb-49f8-4473-8779-67a604e8185a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.730057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8343434e69cf432a8c029e364416047f', 'request_id': 'req-97507ceb-49f8-4473-8779-67a604e8185a'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.734022 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba33a56-e7cd-4678-b032-192f1eda81dd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:02.743556 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] The token cache is being invalidate because project 8343434e69cf432a8c029e364416047f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:02.744060 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.744690 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:02.746471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1256/2480] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:02 2022] DELETE /identity/v3/projects/8343434e69cf432a8c029e364416047f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:02.809720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b7835-f277-41a2-a07c-603eb1be7183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:02.810598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b7835-f277-41a2-a07c-603eb1be7183 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:02.810997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b7835-f277-41a2-a07c-603eb1be7183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:02.811372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b7835-f277-41a2-a07c-603eb1be7183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:02.812272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b7835-f277-41a2-a07c-603eb1be7183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:02.814285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b7835-f277-41a2-a07c-603eb1be7183 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:02.827488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1225/2481] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:02.831728 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:02.833355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:02.843444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:02.849534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97507ceb-49f8-4473-8779-67a604e8185a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:02.856860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcf202-7f1e-48f6-a6f0-b03193c6e6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:02.857978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcf202-7f1e-48f6-a6f0-b03193c6e6e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:02.858358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcf202-7f1e-48f6-a6f0-b03193c6e6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:02.858762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcf202-7f1e-48f6-a6f0-b03193c6e6e6 admin admin] PATH_INFO: `/v3/projects/5c66eb9ee584492e9ac39f413e936343/users/e4d7175579a14dd18f6db4a3a76f9f52/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:02.885801 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcf202-7f1e-48f6-a6f0-b03193c6e6e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c66eb9ee584492e9ac39f413e936343, user_id=e4d7175579a14dd18f6db4a3a76f9f52, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:02.889328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcf202-7f1e-48f6-a6f0-b03193c6e6e6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:02.908667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1257/2482] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:02 2022] PUT /identity/v3/projects/5c66eb9ee584492e9ac39f413e936343/users/e4d7175579a14dd18f6db4a3a76f9f52/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:02.920531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65af43-887d-4206-9b27-f2da59c7e2d1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:02.920822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65af43-887d-4206-9b27-f2da59c7e2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:02.921104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65af43-887d-4206-9b27-f2da59c7e2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:02.951268 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-1a65af43-887d-4206-9b27-f2da59c7e2d1 None None] MFA Rules not processed for user `e4d7175579a14dd18f6db4a3a76f9f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:02.952516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1a65af43-887d-4206-9b27-f2da59c7e2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:03.026622 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1226/2483] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:02 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:03.031314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcf202-7f1e-48f6-a6f0-b03193c6e6e6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:03.099474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcf202-7f1e-48f6-a6f0-b03193c6e6e6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:03.104618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bac31-c479-4608-a5cd-c56031946093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:03.105302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bac31-c479-4608-a5cd-c56031946093 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:03.105588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bac31-c479-4608-a5cd-c56031946093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:03.105863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222bac31-c479-4608-a5cd-c56031946093 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:03.106589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222bac31-c479-4608-a5cd-c56031946093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:03.108581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222bac31-c479-4608-a5cd-c56031946093 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:03.140687 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1258/2484] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:03 2022] POST /identity/v3/projects => generated 403 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:03.142243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65af43-887d-4206-9b27-f2da59c7e2d1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:03.153049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65af43-887d-4206-9b27-f2da59c7e2d1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:03.171668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:03.219472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb911ce-f80c-480a-a738-2bdd16048d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:03.220480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb911ce-f80c-480a-a738-2bdd16048d3d admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:03.220480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb911ce-f80c-480a-a738-2bdd16048d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:03.220832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb911ce-f80c-480a-a738-2bdd16048d3d admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:03.221494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb911ce-f80c-480a-a738-2bdd16048d3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:03.223530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb911ce-f80c-480a-a738-2bdd16048d3d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:03.250531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1227/2485] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:03 2022] POST /identity/v3/users => generated 308 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:03.251762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bac31-c479-4608-a5cd-c56031946093 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:03.262248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222bac31-c479-4608-a5cd-c56031946093 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:03.273178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a17d-2122-45f8-944a-159fc0e442ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:03.273860 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a17d-2122-45f8-944a-159fc0e442ef admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:03.274140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a17d-2122-45f8-944a-159fc0e442ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:03.274412 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a17d-2122-45f8-944a-159fc0e442ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:03.275187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631a17d-2122-45f8-944a-159fc0e442ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:03.276799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631a17d-2122-45f8-944a-159fc0e442ef admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:03.291667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb911ce-f80c-480a-a738-2bdd16048d3d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:03.293456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ccb911ce-f80c-480a-a738-2bdd16048d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:03.298725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1259/2486] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:03.299961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a17d-2122-45f8-944a-159fc0e442ef admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:03.309961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631a17d-2122-45f8-944a-159fc0e442ef admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:03.314606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c20041-3629-41bf-be15-a1be23c8840c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:03.315673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c20041-3629-41bf-be15-a1be23c8840c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:03.315953 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c20041-3629-41bf-be15-a1be23c8840c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:03.316227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c20041-3629-41bf-be15-a1be23c8840c admin admin] PATH_INFO: `/v3/projects/2840a4f57ae14f2588210827575fe00f/users/7014c76527624cf48176c7a272ae4588/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:03.332510 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c20041-3629-41bf-be15-a1be23c8840c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2840a4f57ae14f2588210827575fe00f, user_id=7014c76527624cf48176c7a272ae4588, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:03.335242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c20041-3629-41bf-be15-a1be23c8840c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:03.352024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1260/2487] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:03 2022] PUT /identity/v3/projects/2840a4f57ae14f2588210827575fe00f/users/7014c76527624cf48176c7a272ae4588/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:03.356311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c20041-3629-41bf-be15-a1be23c8840c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:03.369047 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb911ce-f80c-480a-a738-2bdd16048d3d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:03.417145 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c20041-3629-41bf-be15-a1be23c8840c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:03.421218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3a563-d12b-40e2-b267-9ccc1e318b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:03.421884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3a563-d12b-40e2-b267-9ccc1e318b80 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:03.422025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a0a6a-957c-4e39-9fb2-183ebad4b76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:03.422174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3a563-d12b-40e2-b267-9ccc1e318b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:03.422457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3a563-d12b-40e2-b267-9ccc1e318b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:03.422795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a0a6a-957c-4e39-9fb2-183ebad4b76f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:03.423089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a0a6a-957c-4e39-9fb2-183ebad4b76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:03.423370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a0a6a-957c-4e39-9fb2-183ebad4b76f admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:03.424106 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a0a6a-957c-4e39-9fb2-183ebad4b76f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:03.424517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d9f3a563-d12b-40e2-b267-9ccc1e318b80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:03.425788 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a0a6a-957c-4e39-9fb2-183ebad4b76f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:03.436018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1261/2488] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:03.437848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3a563-d12b-40e2-b267-9ccc1e318b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:03.439343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3a563-d12b-40e2-b267-9ccc1e318b80 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:03.440189 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a0a6a-957c-4e39-9fb2-183ebad4b76f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:03.449558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1228/2489] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:49:03.449716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a0a6a-957c-4e39-9fb2-183ebad4b76f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:03.459763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27093c72-0214-467f-a40a-999d96426ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:03.460774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27093c72-0214-467f-a40a-999d96426ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:03.461058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27093c72-0214-467f-a40a-999d96426ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:03.461377 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27093c72-0214-467f-a40a-999d96426ccf admin admin] PATH_INFO: `/v3/projects/2840a4f57ae14f2588210827575fe00f/users/7014c76527624cf48176c7a272ae4588/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:03.463385 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27093c72-0214-467f-a40a-999d96426ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2840a4f57ae14f2588210827575fe00f, user_id=7014c76527624cf48176c7a272ae4588, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:03.466464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27093c72-0214-467f-a40a-999d96426ccf admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:03.477255 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1262/2490] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:03 2022] PUT /identity/v3/projects/2840a4f57ae14f2588210827575fe00f/users/7014c76527624cf48176c7a272ae4588/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:03.488025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33dfe8-eb44-4f61-8e7d-aa7b76821614 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:03.488560 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33dfe8-eb44-4f61-8e7d-aa7b76821614 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:03.488669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33dfe8-eb44-4f61-8e7d-aa7b76821614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:03.515674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-eb33dfe8-eb44-4f61-8e7d-aa7b76821614 None None] MFA Rules not processed for user `7014c76527624cf48176c7a272ae4588`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:03.516876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-eb33dfe8-eb44-4f61-8e7d-aa7b76821614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:03.564724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:03.587488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1229/2491] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:03 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:03.598809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27093c72-0214-467f-a40a-999d96426ccf admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:03.600750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-27093c72-0214-467f-a40a-999d96426ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:03.616306 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:03.689824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27093c72-0214-467f-a40a-999d96426ccf admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:03.694307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1b92e-7612-4c6e-8ba0-7d3728ef693a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:03.695036 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1b92e-7612-4c6e-8ba0-7d3728ef693a service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:03.695323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1b92e-7612-4c6e-8ba0-7d3728ef693a service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:03.695631 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1b92e-7612-4c6e-8ba0-7d3728ef693a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:03.706469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1b92e-7612-4c6e-8ba0-7d3728ef693a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:03.707977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1b92e-7612-4c6e-8ba0-7d3728ef693a service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:03.718998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1263/2492] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:03 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:03.878080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33dfe8-eb44-4f61-8e7d-aa7b76821614 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:03.943328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33dfe8-eb44-4f61-8e7d-aa7b76821614 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:03.948177 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:03.949064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:03.949342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:03.949707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] PATH_INFO: `/v3/users/e4d7175579a14dd18f6db4a3a76f9f52` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:03.950926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d7175579a14dd18f6db4a3a76f9f52)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:03.953083 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:04.007269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d7175579a14dd18f6db4a3a76f9f52'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.007959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d7175579a14dd18f6db4a3a76f9f52'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.021246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d7175579a14dd18f6db4a3a76f9f52'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.021917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d7175579a14dd18f6db4a3a76f9f52'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.030158 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d7175579a14dd18f6db4a3a76f9f52'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.043550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1230/2493] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:03 2022] DELETE /identity/v3/users/e4d7175579a14dd18f6db4a3a76f9f52 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:04.048248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc22cf4-2aeb-4d78-ba73-7c9328224233 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:04.048748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc22cf4-2aeb-4d78-ba73-7c9328224233 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:04.049028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc22cf4-2aeb-4d78-ba73-7c9328224233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:04.067002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-5cc22cf4-2aeb-4d78-ba73-7c9328224233 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:04.068206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5cc22cf4-2aeb-4d78-ba73-7c9328224233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:04.127871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1264/2494] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:04.138161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:04.139692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:04.200840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434a577-1049-4c5f-a6e9-87c6950f6509 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:04.204765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f3778-a730-45b7-afb4-11293d39eec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:04.205427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3778-a730-45b7-afb4-11293d39eec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:04.205708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3778-a730-45b7-afb4-11293d39eec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:04.205984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3778-a730-45b7-afb4-11293d39eec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:04.216402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f3778-a730-45b7-afb4-11293d39eec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:04.217837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f3778-a730-45b7-afb4-11293d39eec6 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:04.228113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1231/2495] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:04.687762 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc22cf4-2aeb-4d78-ba73-7c9328224233 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:04.689263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5cc22cf4-2aeb-4d78-ba73-7c9328224233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:04.701969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc22cf4-2aeb-4d78-ba73-7c9328224233 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:04.708277 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:04.709355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:04.709765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:04.710133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] PATH_INFO: `/v3/projects/5c66eb9ee584492e9ac39f413e936343` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:04.711462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c66eb9ee584492e9ac39f413e936343)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:04.713560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:04.774286 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c66eb9ee584492e9ac39f413e936343', 'request_id': 'req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.781116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c66eb9ee584492e9ac39f413e936343', 'request_id': 'req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.787824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c66eb9ee584492e9ac39f413e936343', 'request_id': 'req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.804265 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] The token cache is being invalidate because project 5c66eb9ee584492e9ac39f413e936343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:04.804568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.805252 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.807226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1265/2496] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:04 2022] DELETE /identity/v3/projects/5c66eb9ee584492e9ac39f413e936343 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:04.811053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f3778-a730-45b7-afb4-11293d39eec6 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:04.828507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f3778-a730-45b7-afb4-11293d39eec6 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:04.835055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:04.916287 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc74111-44c5-486f-93f5-884cf34baf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:04.917188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc74111-44c5-486f-93f5-884cf34baf94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:04.917468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc74111-44c5-486f-93f5-884cf34baf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:04.917740 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc74111-44c5-486f-93f5-884cf34baf94 admin admin] PATH_INFO: `/v3/users/7014c76527624cf48176c7a272ae4588` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:04.919066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc74111-44c5-486f-93f5-884cf34baf94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7014c76527624cf48176c7a272ae4588)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:04.921034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc74111-44c5-486f-93f5-884cf34baf94 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:04.983195 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bcc74111-44c5-486f-93f5-884cf34baf94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7014c76527624cf48176c7a272ae4588'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.983878 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bcc74111-44c5-486f-93f5-884cf34baf94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7014c76527624cf48176c7a272ae4588'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:04.999912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bcc74111-44c5-486f-93f5-884cf34baf94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7014c76527624cf48176c7a272ae4588'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:05.000658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bcc74111-44c5-486f-93f5-884cf34baf94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7014c76527624cf48176c7a272ae4588'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:05.010217 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bcc74111-44c5-486f-93f5-884cf34baf94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7014c76527624cf48176c7a272ae4588'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:05.023920 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1232/2497] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:04 2022] DELETE /identity/v3/users/7014c76527624cf48176c7a272ae4588 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:05.316420 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:05.316790 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:05.350009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5f3b4-d0ff-4680-a5c0-fe63deb1e6ee admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:05.368704 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:05.431680 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:05.433041 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:05.433512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:05.434009 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] PATH_INFO: `/v3/projects/2840a4f57ae14f2588210827575fe00f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:05.435434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2840a4f57ae14f2588210827575fe00f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:05.437588 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:05.499114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2840a4f57ae14f2588210827575fe00f', 'request_id': 'req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:05.506722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2840a4f57ae14f2588210827575fe00f', 'request_id': 'req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:05.514160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2840a4f57ae14f2588210827575fe00f', 'request_id': 'req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:05.541368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] The token cache is being invalidate because project 2840a4f57ae14f2588210827575fe00f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:05.541668 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:05.542369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c3ac4e24-3f92-47f6-bb04-0d5f3b132f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:05.547827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1266/2498] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:05 2022] DELETE /identity/v3/projects/2840a4f57ae14f2588210827575fe00f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:05.549670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e70d2-844e-44c4-8eae-429cdf160046 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:05.550179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e70d2-844e-44c4-8eae-429cdf160046 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:05.550502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e70d2-844e-44c4-8eae-429cdf160046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:05.572018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-331e70d2-844e-44c4-8eae-429cdf160046 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:05.573876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-331e70d2-844e-44c4-8eae-429cdf160046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:05.645076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1233/2499] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:05.652623 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa6989-6e06-4697-b828-a5eb6d4cbaab None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:05.653205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa6989-6e06-4697-b828-a5eb6d4cbaab None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:05.653493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa6989-6e06-4697-b828-a5eb6d4cbaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:05.673712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ebfa6989-6e06-4697-b828-a5eb6d4cbaab None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:05.674805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ebfa6989-6e06-4697-b828-a5eb6d4cbaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:05.684013 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1267/2500] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:05.688459 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e70d2-844e-44c4-8eae-429cdf160046 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:05.701650 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e70d2-844e-44c4-8eae-429cdf160046 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:05.708489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d0e35-2659-4d3a-a0ff-3aabb1013d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:05.709575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d0e35-2659-4d3a-a0ff-3aabb1013d27 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:05.710014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d0e35-2659-4d3a-a0ff-3aabb1013d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:05.710460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d0e35-2659-4d3a-a0ff-3aabb1013d27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:05.711385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d0e35-2659-4d3a-a0ff-3aabb1013d27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:05.711844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d0e35-2659-4d3a-a0ff-3aabb1013d27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:05.713785 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d0e35-2659-4d3a-a0ff-3aabb1013d27 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:05.730992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1234/2501] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:49:05.735813 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa6989-6e06-4697-b828-a5eb6d4cbaab None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:05.757158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa6989-6e06-4697-b828-a5eb6d4cbaab None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:05.761699 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bb35e-d887-4322-abc4-1170e1b1be2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:05.762397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bb35e-d887-4322-abc4-1170e1b1be2a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:05.762678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bb35e-d887-4322-abc4-1170e1b1be2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:05.763336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bb35e-d887-4322-abc4-1170e1b1be2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:05.763704 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bb35e-d887-4322-abc4-1170e1b1be2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:05.765721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bb35e-d887-4322-abc4-1170e1b1be2a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:05.796203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1268/2502] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:05 2022] POST /identity/v3/projects => generated 355 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:05.800723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d0e35-2659-4d3a-a0ff-3aabb1013d27 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:05.811435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d0e35-2659-4d3a-a0ff-3aabb1013d27 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:05.820750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:05.875174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563e949-1500-4470-a126-cb5007cdedbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:05.876017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e949-1500-4470-a126-cb5007cdedbd admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:05.876394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e949-1500-4470-a126-cb5007cdedbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:05.876770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563e949-1500-4470-a126-cb5007cdedbd admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:05.877688 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563e949-1500-4470-a126-cb5007cdedbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:05.879746 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563e949-1500-4470-a126-cb5007cdedbd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:05.903167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1235/2503] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:05 2022] POST /identity/v3/users => generated 278 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:05.909973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bb35e-d887-4322-abc4-1170e1b1be2a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:05.923334 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bb35e-d887-4322-abc4-1170e1b1be2a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:05.929559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e9213-fa51-41e7-825d-91fa8132ebd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:05.930359 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e9213-fa51-41e7-825d-91fa8132ebd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:05.930772 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e9213-fa51-41e7-825d-91fa8132ebd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:05.931123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e9213-fa51-41e7-825d-91fa8132ebd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:05.931940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e9213-fa51-41e7-825d-91fa8132ebd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:05.933804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e9213-fa51-41e7-825d-91fa8132ebd8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:05.947639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1269/2504] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:05.953685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563e949-1500-4470-a126-cb5007cdedbd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:05.964985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563e949-1500-4470-a126-cb5007cdedbd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:05.969626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d6780-2c4b-4e86-9d8b-b7c97d0ee78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:05.970398 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d6780-2c4b-4e86-9d8b-b7c97d0ee78f admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:05.970730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d6780-2c4b-4e86-9d8b-b7c97d0ee78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:05.971006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d6780-2c4b-4e86-9d8b-b7c97d0ee78f admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:05.972488 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d6780-2c4b-4e86-9d8b-b7c97d0ee78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:05.974277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d6780-2c4b-4e86-9d8b-b7c97d0ee78f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:05.991515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1236/2505] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:05.992945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e9213-fa51-41e7-825d-91fa8132ebd8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:06.003479 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e9213-fa51-41e7-825d-91fa8132ebd8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:06.008950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eb973-8488-4441-acae-de7c5701165f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:06.009635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eb973-8488-4441-acae-de7c5701165f admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:06.009918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eb973-8488-4441-acae-de7c5701165f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:06.010199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eb973-8488-4441-acae-de7c5701165f admin admin] PATH_INFO: `/v3/projects/f6c9fe1eae84418680169a11a739ab60/users/f5a46d3150ec496ebe7342def072e7fc/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:06.026261 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927eb973-8488-4441-acae-de7c5701165f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c9fe1eae84418680169a11a739ab60, user_id=f5a46d3150ec496ebe7342def072e7fc, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:06.028982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927eb973-8488-4441-acae-de7c5701165f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:06.044197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1270/2506] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:05 2022] PUT /identity/v3/projects/f6c9fe1eae84418680169a11a739ab60/users/f5a46d3150ec496ebe7342def072e7fc/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:06.055852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d3514-1197-4a17-9f81-c751b19703a1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:06.056140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d3514-1197-4a17-9f81-c751b19703a1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:06.056414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d3514-1197-4a17-9f81-c751b19703a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:06.083805 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-874d3514-1197-4a17-9f81-c751b19703a1 None None] MFA Rules not processed for user `f5a46d3150ec496ebe7342def072e7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:06.085027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-874d3514-1197-4a17-9f81-c751b19703a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:06.150985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1237/2507] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:06.155622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eb973-8488-4441-acae-de7c5701165f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:06.203789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:06.227502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eb973-8488-4441-acae-de7c5701165f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:06.244309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87737d3-31d7-4f9d-8154-c1b1f6cbf745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:06.245276 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87737d3-31d7-4f9d-8154-c1b1f6cbf745 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:06.245727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87737d3-31d7-4f9d-8154-c1b1f6cbf745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:06.246184 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87737d3-31d7-4f9d-8154-c1b1f6cbf745 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:06.247755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87737d3-31d7-4f9d-8154-c1b1f6cbf745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:06.250537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87737d3-31d7-4f9d-8154-c1b1f6cbf745 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:06.288561 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1271/2508] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:06 2022] POST /identity/v3/projects => generated 357 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:06.292261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d3514-1197-4a17-9f81-c751b19703a1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:06.304190 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d3514-1197-4a17-9f81-c751b19703a1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:06.372467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab0b4f-36cf-4328-aaab-1267878f19ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:06.373162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab0b4f-36cf-4328-aaab-1267878f19ed admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:06.373448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab0b4f-36cf-4328-aaab-1267878f19ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:06.373727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab0b4f-36cf-4328-aaab-1267878f19ed admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:06.374568 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab0b4f-36cf-4328-aaab-1267878f19ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:06.376647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab0b4f-36cf-4328-aaab-1267878f19ed admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:06.398752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1238/2509] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:06 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:06.402799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87737d3-31d7-4f9d-8154-c1b1f6cbf745 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:06.413798 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87737d3-31d7-4f9d-8154-c1b1f6cbf745 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:06.418608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed9e46-e12b-45b7-8151-49fc47140f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:06.419302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed9e46-e12b-45b7-8151-49fc47140f3d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:06.419581 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed9e46-e12b-45b7-8151-49fc47140f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:06.419839 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed9e46-e12b-45b7-8151-49fc47140f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:06.420587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed9e46-e12b-45b7-8151-49fc47140f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:06.422316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed9e46-e12b-45b7-8151-49fc47140f3d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:06.433456 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1272/2510] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:06.437700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab0b4f-36cf-4328-aaab-1267878f19ed admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:06.449233 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab0b4f-36cf-4328-aaab-1267878f19ed admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:06.454683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3388c-0516-4734-b034-ca3fcd1dc195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:06.455762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3388c-0516-4734-b034-ca3fcd1dc195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:06.456148 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3388c-0516-4734-b034-ca3fcd1dc195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:06.456467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3388c-0516-4734-b034-ca3fcd1dc195 admin admin] PATH_INFO: `/v3/projects/2a4363bad0e341868e161e187cc99041/users/cd8b8c2e03ea4ed28599a71ba4c50068/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:06.473247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:06.481641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3388c-0516-4734-b034-ca3fcd1dc195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4363bad0e341868e161e187cc99041, user_id=cd8b8c2e03ea4ed28599a71ba4c50068, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:06.484653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3388c-0516-4734-b034-ca3fcd1dc195 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:06.510346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1239/2511] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:06 2022] PUT /identity/v3/projects/2a4363bad0e341868e161e187cc99041/users/cd8b8c2e03ea4ed28599a71ba4c50068/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:06.514444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed9e46-e12b-45b7-8151-49fc47140f3d admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:06.527029 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed9e46-e12b-45b7-8151-49fc47140f3d admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:06.601440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab0fa5-019f-4e45-975f-b804e7f6b1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:06.602198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0fa5-019f-4e45-975f-b804e7f6b1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:06.602543 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0fa5-019f-4e45-975f-b804e7f6b1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:06.602931 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0fa5-019f-4e45-975f-b804e7f6b1f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:06.603789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab0fa5-019f-4e45-975f-b804e7f6b1f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:06.605788 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab0fa5-019f-4e45-975f-b804e7f6b1f0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:06.619607 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1273/2512] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:06.623806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3388c-0516-4734-b034-ca3fcd1dc195 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:06.635750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3388c-0516-4734-b034-ca3fcd1dc195 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:06.640616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c0776-3fe2-495d-b56b-b155cc246619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:06.641442 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c0776-3fe2-495d-b56b-b155cc246619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:06.641733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c0776-3fe2-495d-b56b-b155cc246619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:06.641996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837c0776-3fe2-495d-b56b-b155cc246619 admin admin] PATH_INFO: `/v3/projects/2a4363bad0e341868e161e187cc99041/users/cd8b8c2e03ea4ed28599a71ba4c50068/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:06.644046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c0776-3fe2-495d-b56b-b155cc246619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4363bad0e341868e161e187cc99041, user_id=cd8b8c2e03ea4ed28599a71ba4c50068, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:06.646685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837c0776-3fe2-495d-b56b-b155cc246619 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:06.669006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1240/2513] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:06 2022] PUT /identity/v3/projects/2a4363bad0e341868e161e187cc99041/users/cd8b8c2e03ea4ed28599a71ba4c50068/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:06.679780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aaae2-d409-4f81-b7df-6c7ed6a96088 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:06.682373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aaae2-d409-4f81-b7df-6c7ed6a96088 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:06.682653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aaae2-d409-4f81-b7df-6c7ed6a96088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:06.723614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0e0aaae2-d409-4f81-b7df-6c7ed6a96088 None None] MFA Rules not processed for user `cd8b8c2e03ea4ed28599a71ba4c50068`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:06.724361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0e0aaae2-d409-4f81-b7df-6c7ed6a96088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:06.825164 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1274/2514] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:06 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:06.845642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c0776-3fe2-495d-b56b-b155cc246619 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:06.849657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-837c0776-3fe2-495d-b56b-b155cc246619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:06.963105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837c0776-3fe2-495d-b56b-b155cc246619 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:06.968424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcad82f-4b78-45d4-8814-45eac7b4da65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:06.969225 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcad82f-4b78-45d4-8814-45eac7b4da65 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:06.969625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcad82f-4b78-45d4-8814-45eac7b4da65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:06.970337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcad82f-4b78-45d4-8814-45eac7b4da65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:06.985558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcad82f-4b78-45d4-8814-45eac7b4da65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:06.987390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcad82f-4b78-45d4-8814-45eac7b4da65 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:07.013119 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1241/2515] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:06 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:07.129188 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0aaae2-d409-4f81-b7df-6c7ed6a96088 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:07.139274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcad82f-4b78-45d4-8814-45eac7b4da65 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:07.140707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-efcad82f-4b78-45d4-8814-45eac7b4da65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:07.142879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0aaae2-d409-4f81-b7df-6c7ed6a96088 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:07.146765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174788-eb11-48e9-ba6d-2d2ab8ee5ed9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:07.147554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174788-eb11-48e9-ba6d-2d2ab8ee5ed9 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:07.147894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174788-eb11-48e9-ba6d-2d2ab8ee5ed9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:07.148231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174788-eb11-48e9-ba6d-2d2ab8ee5ed9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:07.206682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:07.221944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcad82f-4b78-45d4-8814-45eac7b4da65 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:07.226478 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fad07-3c69-4453-88e4-450ecbb70cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:07.227401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fad07-3c69-4453-88e4-450ecbb70cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:07.227678 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fad07-3c69-4453-88e4-450ecbb70cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:07.227948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fad07-3c69-4453-88e4-450ecbb70cfa admin admin] PATH_INFO: `/v3/users/bccf002d9da34d1dad0ee71b5b7bb1c1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:07.228182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c174788-eb11-48e9-ba6d-2d2ab8ee5ed9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:07.229135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fad07-3c69-4453-88e4-450ecbb70cfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=bccf002d9da34d1dad0ee71b5b7bb1c1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:07.229634 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c174788-eb11-48e9-ba6d-2d2ab8ee5ed9 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:07.231082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fad07-3c69-4453-88e4-450ecbb70cfa admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:07.238567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1275/2516] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:07 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:07.290938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-466fad07-3c69-4453-88e4-450ecbb70cfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bccf002d9da34d1dad0ee71b5b7bb1c1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:07.291832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-466fad07-3c69-4453-88e4-450ecbb70cfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bccf002d9da34d1dad0ee71b5b7bb1c1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:07.306760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-466fad07-3c69-4453-88e4-450ecbb70cfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bccf002d9da34d1dad0ee71b5b7bb1c1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:07.307662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-466fad07-3c69-4453-88e4-450ecbb70cfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bccf002d9da34d1dad0ee71b5b7bb1c1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:07.326837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-466fad07-3c69-4453-88e4-450ecbb70cfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bccf002d9da34d1dad0ee71b5b7bb1c1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:07.332936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:07.342870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1242/2517] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:07 2022] DELETE /identity/v3/users/bccf002d9da34d1dad0ee71b5b7bb1c1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:07.541669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174788-eb11-48e9-ba6d-2d2ab8ee5ed9 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:07.553310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174788-eb11-48e9-ba6d-2d2ab8ee5ed9 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:07.612088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:07.612956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:07.613240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:07.613517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] PATH_INFO: `/v3/projects/8f94b91ae9ee464aa2f5e24f5e8e0f7b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:07.614722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f94b91ae9ee464aa2f5e24f5e8e0f7b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:07.616661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:07.675184 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f94b91ae9ee464aa2f5e24f5e8e0f7b', 'request_id': 'req-fbdc5729-324b-4962-aa0c-9aadffcede9c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:07.681506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f94b91ae9ee464aa2f5e24f5e8e0f7b', 'request_id': 'req-fbdc5729-324b-4962-aa0c-9aadffcede9c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:07.687353 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f94b91ae9ee464aa2f5e24f5e8e0f7b', 'request_id': 'req-fbdc5729-324b-4962-aa0c-9aadffcede9c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:07.701984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] The token cache is being invalidate because project 8f94b91ae9ee464aa2f5e24f5e8e0f7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:07.702269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:07.702928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fbdc5729-324b-4962-aa0c-9aadffcede9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:07.704510 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1276/2518] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:07 2022] DELETE /identity/v3/projects/8f94b91ae9ee464aa2f5e24f5e8e0f7b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:07.709839 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cafa5-b9f9-412c-983a-6f48acbeeb6a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:07.710152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cafa5-b9f9-412c-983a-6f48acbeeb6a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:07.710432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cafa5-b9f9-412c-983a-6f48acbeeb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:07.729215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9e9cafa5-b9f9-412c-983a-6f48acbeeb6a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:07.730421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9e9cafa5-b9f9-412c-983a-6f48acbeeb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:07.794104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1243/2519] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:07.806477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddece10e-7db0-405d-8ad2-8e5da88fe0cc None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:07.806899 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddece10e-7db0-405d-8ad2-8e5da88fe0cc None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:07.807207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddece10e-7db0-405d-8ad2-8e5da88fe0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:07.829583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ddece10e-7db0-405d-8ad2-8e5da88fe0cc None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:07.830904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ddece10e-7db0-405d-8ad2-8e5da88fe0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:07.840469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1277/2520] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:07.844796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cafa5-b9f9-412c-983a-6f48acbeeb6a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:07.855667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cafa5-b9f9-412c-983a-6f48acbeeb6a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:07.861312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fadf9-773e-4d99-981f-eac6991c2efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:07.862057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fadf9-773e-4d99-981f-eac6991c2efc admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:07.862337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fadf9-773e-4d99-981f-eac6991c2efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:07.862619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fadf9-773e-4d99-981f-eac6991c2efc admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:07.863391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fadf9-773e-4d99-981f-eac6991c2efc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:07.863704 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fadf9-773e-4d99-981f-eac6991c2efc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:07.865324 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fadf9-773e-4d99-981f-eac6991c2efc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:07.877492 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1244/2521] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:49:07.883955 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddece10e-7db0-405d-8ad2-8e5da88fe0cc None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:07.899180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddece10e-7db0-405d-8ad2-8e5da88fe0cc None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:07.908528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5b33b-2465-4566-9595-decb22c10e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:07.909317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5b33b-2465-4566-9595-decb22c10e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:07.909563 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5b33b-2465-4566-9595-decb22c10e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:07.909834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5b33b-2465-4566-9595-decb22c10e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:07.910533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5b33b-2465-4566-9595-decb22c10e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:07.912480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb5b33b-2465-4566-9595-decb22c10e16 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:07.942443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1278/2522] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:07 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:07.947065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fadf9-773e-4d99-981f-eac6991c2efc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:07.957936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fadf9-773e-4d99-981f-eac6991c2efc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:08.018373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f42188-3029-40e2-9485-565b43568213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:08.019074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f42188-3029-40e2-9485-565b43568213 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:08.019361 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f42188-3029-40e2-9485-565b43568213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:08.019643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f42188-3029-40e2-9485-565b43568213 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:08.020366 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f42188-3029-40e2-9485-565b43568213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:08.022328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f42188-3029-40e2-9485-565b43568213 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:08.053359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1245/2523] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:07 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:08.054972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5b33b-2465-4566-9595-decb22c10e16 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:08.067081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5b33b-2465-4566-9595-decb22c10e16 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:08.073054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd252928-60d7-4f4f-b266-1dbeac56d8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:08.073731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd252928-60d7-4f4f-b266-1dbeac56d8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:08.074008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd252928-60d7-4f4f-b266-1dbeac56d8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:08.074286 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd252928-60d7-4f4f-b266-1dbeac56d8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:08.075055 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd252928-60d7-4f4f-b266-1dbeac56d8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:08.076686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd252928-60d7-4f4f-b266-1dbeac56d8f4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:08.088413 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1279/2524] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:08.094204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f42188-3029-40e2-9485-565b43568213 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:08.107302 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f42188-3029-40e2-9485-565b43568213 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:08.112210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf4955-ff4b-4e12-af67-8074b5ed0642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:08.112812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf4955-ff4b-4e12-af67-8074b5ed0642 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:08.113088 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf4955-ff4b-4e12-af67-8074b5ed0642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:08.113363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf4955-ff4b-4e12-af67-8074b5ed0642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:08.114115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf4955-ff4b-4e12-af67-8074b5ed0642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:08.115748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf4955-ff4b-4e12-af67-8074b5ed0642 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:08.126408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1246/2525] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:08.130667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd252928-60d7-4f4f-b266-1dbeac56d8f4 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:08.141239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd252928-60d7-4f4f-b266-1dbeac56d8f4 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:08.146644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac14633-f9cf-4e18-a104-3396425d0407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:08.147567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac14633-f9cf-4e18-a104-3396425d0407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:08.147852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac14633-f9cf-4e18-a104-3396425d0407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:08.148124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac14633-f9cf-4e18-a104-3396425d0407 admin admin] PATH_INFO: `/v3/projects/a3033741be8842ca884bf6580397d453/users/e24730e01a38440b82fb24e4d85a9407/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:08.164287 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac14633-f9cf-4e18-a104-3396425d0407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3033741be8842ca884bf6580397d453, user_id=e24730e01a38440b82fb24e4d85a9407, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:08.167168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac14633-f9cf-4e18-a104-3396425d0407 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:08.183841 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1280/2526] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:08 2022] PUT /identity/v3/projects/a3033741be8842ca884bf6580397d453/users/e24730e01a38440b82fb24e4d85a9407/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:08.194178 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c1789-183c-4929-baec-d23ef7494520 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:08.194538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c1789-183c-4929-baec-d23ef7494520 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:08.194912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c1789-183c-4929-baec-d23ef7494520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:08.227025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-0e7c1789-183c-4929-baec-d23ef7494520 None None] MFA Rules not processed for user `e24730e01a38440b82fb24e4d85a9407`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:08.228240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0e7c1789-183c-4929-baec-d23ef7494520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:08.276294 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:08.295778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1247/2527] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:08.307474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac14633-f9cf-4e18-a104-3396425d0407 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:08.309360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cac14633-f9cf-4e18-a104-3396425d0407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:08.316944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:08.400188 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac14633-f9cf-4e18-a104-3396425d0407 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:08.407913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b71626-1a0b-4407-a9c4-13c16e52e005 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:08.408568 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b71626-1a0b-4407-a9c4-13c16e52e005 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:08.408880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b71626-1a0b-4407-a9c4-13c16e52e005 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:08.409160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b71626-1a0b-4407-a9c4-13c16e52e005 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:08.419173 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b71626-1a0b-4407-a9c4-13c16e52e005 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:08.420745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b71626-1a0b-4407-a9c4-13c16e52e005 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:08.430329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1281/2528] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:08.653776 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c1789-183c-4929-baec-d23ef7494520 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:08.723443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c1789-183c-4929-baec-d23ef7494520 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:08.728768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:08.729655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:08.729937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:08.730214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] PATH_INFO: `/v3/users/e24730e01a38440b82fb24e4d85a9407` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:08.731449 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e24730e01a38440b82fb24e4d85a9407)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:08.733378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:08.783609 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e24730e01a38440b82fb24e4d85a9407'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:08.784444 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e24730e01a38440b82fb24e4d85a9407'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:08.797340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e24730e01a38440b82fb24e4d85a9407'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:08.798041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e24730e01a38440b82fb24e4d85a9407'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:08.807300 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e24730e01a38440b82fb24e4d85a9407'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:08.820105 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1248/2529] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:08 2022] DELETE /identity/v3/users/e24730e01a38440b82fb24e4d85a9407 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:08.824710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227e64c-b174-4129-92a0-61f7a7ad9ba9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:08.825004 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227e64c-b174-4129-92a0-61f7a7ad9ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:08.825285 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227e64c-b174-4129-92a0-61f7a7ad9ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:08.843709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0227e64c-b174-4129-92a0-61f7a7ad9ba9 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:08.844893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0227e64c-b174-4129-92a0-61f7a7ad9ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:08.906724 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1282/2530] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:08.917170 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:08.918671 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:08.983434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70ad9f-9df2-4147-9c41-64194a40fbdd admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:08.988664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a3893-9592-4a7f-ac57-cb4f0adefa5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:08.989633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a3893-9592-4a7f-ac57-cb4f0adefa5f service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:08.990052 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a3893-9592-4a7f-ac57-cb4f0adefa5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:08.990456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a3893-9592-4a7f-ac57-cb4f0adefa5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:09.006764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a3893-9592-4a7f-ac57-cb4f0adefa5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:09.009092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a3893-9592-4a7f-ac57-cb4f0adefa5f service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:09.022662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1249/2531] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:09.333019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227e64c-b174-4129-92a0-61f7a7ad9ba9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:09.334539 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0227e64c-b174-4129-92a0-61f7a7ad9ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:09.363430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227e64c-b174-4129-92a0-61f7a7ad9ba9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:09.370957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:09.372656 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:09.372913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:09.373222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] PATH_INFO: `/v3/projects/a3033741be8842ca884bf6580397d453` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:09.374632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3033741be8842ca884bf6580397d453)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:09.377098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:09.430875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3033741be8842ca884bf6580397d453', 'request_id': 'req-d8165dc7-d414-422e-819e-6813fa30a462'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:09.440054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3033741be8842ca884bf6580397d453', 'request_id': 'req-d8165dc7-d414-422e-819e-6813fa30a462'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:09.447577 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3033741be8842ca884bf6580397d453', 'request_id': 'req-d8165dc7-d414-422e-819e-6813fa30a462'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:09.462194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] The token cache is being invalidate because project a3033741be8842ca884bf6580397d453 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:09.462483 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:09.463290 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d8165dc7-d414-422e-819e-6813fa30a462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:09.465212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1283/2532] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:09 2022] DELETE /identity/v3/projects/a3033741be8842ca884bf6580397d453 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:09.472328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde01091-1eed-431e-9489-0530142d18a0 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:09.472614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde01091-1eed-431e-9489-0530142d18a0 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:09.472889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde01091-1eed-431e-9489-0530142d18a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:09.492992 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cde01091-1eed-431e-9489-0530142d18a0 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:09.494188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cde01091-1eed-431e-9489-0530142d18a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:09.503450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:09.558480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1250/2533] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:09.570521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347fe8c-2c67-46ca-9d17-b645d43f43b8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:09.570930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347fe8c-2c67-46ca-9d17-b645d43f43b8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:09.571297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347fe8c-2c67-46ca-9d17-b645d43f43b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:09.594615 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2347fe8c-2c67-46ca-9d17-b645d43f43b8 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:09.595849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2347fe8c-2c67-46ca-9d17-b645d43f43b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:09.606851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1284/2534] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:09.611229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde01091-1eed-431e-9489-0530142d18a0 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:09.621211 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde01091-1eed-431e-9489-0530142d18a0 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:09.625962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473f2aa-b212-4436-abea-540bcdff75e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:09.626623 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473f2aa-b212-4436-abea-540bcdff75e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:09.626933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473f2aa-b212-4436-abea-540bcdff75e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:09.627208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473f2aa-b212-4436-abea-540bcdff75e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:09.627938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473f2aa-b212-4436-abea-540bcdff75e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:09.628240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473f2aa-b212-4436-abea-540bcdff75e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:09.630112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473f2aa-b212-4436-abea-540bcdff75e7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:09.647882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1251/2535] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:49:09.652207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347fe8c-2c67-46ca-9d17-b645d43f43b8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:09.661807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347fe8c-2c67-46ca-9d17-b645d43f43b8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:09.668003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57771e7-a8a0-4828-b099-782e809a47df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:09.669182 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57771e7-a8a0-4828-b099-782e809a47df admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:09.669639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57771e7-a8a0-4828-b099-782e809a47df admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:09.669994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57771e7-a8a0-4828-b099-782e809a47df admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:09.670712 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57771e7-a8a0-4828-b099-782e809a47df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:09.672800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57771e7-a8a0-4828-b099-782e809a47df admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:09.676323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:09.710516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1285/2536] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:09 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:09.715204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473f2aa-b212-4436-abea-540bcdff75e7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:09.726133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473f2aa-b212-4436-abea-540bcdff75e7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:09.790994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55244b-548c-4286-9b20-6cccd6c91da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:09.791845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55244b-548c-4286-9b20-6cccd6c91da0 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:09.792124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55244b-548c-4286-9b20-6cccd6c91da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:09.792399 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55244b-548c-4286-9b20-6cccd6c91da0 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:09.793137 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55244b-548c-4286-9b20-6cccd6c91da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:09.796086 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55244b-548c-4286-9b20-6cccd6c91da0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:09.819898 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1252/2537] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:09 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:09.824445 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57771e7-a8a0-4828-b099-782e809a47df admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:09.835328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57771e7-a8a0-4828-b099-782e809a47df admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:09.840760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018730c-7c0f-4758-b3d4-d16c7d5af641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:09.841465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018730c-7c0f-4758-b3d4-d16c7d5af641 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:09.841746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018730c-7c0f-4758-b3d4-d16c7d5af641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:09.842018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a018730c-7c0f-4758-b3d4-d16c7d5af641 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:09.842815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018730c-7c0f-4758-b3d4-d16c7d5af641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:09.844421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a018730c-7c0f-4758-b3d4-d16c7d5af641 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:09.856032 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1286/2538] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:09.860439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55244b-548c-4286-9b20-6cccd6c91da0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:09.870944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55244b-548c-4286-9b20-6cccd6c91da0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:09.876944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f4197-926d-4c07-bd9f-0e6a338330ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:09.877765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f4197-926d-4c07-bd9f-0e6a338330ab admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:09.878050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f4197-926d-4c07-bd9f-0e6a338330ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:09.879018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f4197-926d-4c07-bd9f-0e6a338330ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:09.880065 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f4197-926d-4c07-bd9f-0e6a338330ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:09.882058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f4197-926d-4c07-bd9f-0e6a338330ab admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:09.896683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1253/2539] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:09.897737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018730c-7c0f-4758-b3d4-d16c7d5af641 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:09.908041 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a018730c-7c0f-4758-b3d4-d16c7d5af641 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:09.913036 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dae9a-6d5d-43f9-a17f-166adb48415b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:09.913941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dae9a-6d5d-43f9-a17f-166adb48415b admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:09.914223 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dae9a-6d5d-43f9-a17f-166adb48415b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:09.914494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dae9a-6d5d-43f9-a17f-166adb48415b admin admin] PATH_INFO: `/v3/projects/c0780588ce2d4e73a651a310891c9322/users/1854af616f344b2c9cb6ab4e5c022ed4/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:09.935010 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dae9a-6d5d-43f9-a17f-166adb48415b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0780588ce2d4e73a651a310891c9322, user_id=1854af616f344b2c9cb6ab4e5c022ed4, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:09.945726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dae9a-6d5d-43f9-a17f-166adb48415b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:09.957512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1287/2540] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:09 2022] PUT /identity/v3/projects/c0780588ce2d4e73a651a310891c9322/users/1854af616f344b2c9cb6ab4e5c022ed4/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:09.967999 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a126be-89fc-4b44-9302-0c8c32564a5c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:09.968815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a126be-89fc-4b44-9302-0c8c32564a5c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:09.969642 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a126be-89fc-4b44-9302-0c8c32564a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:10.001763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-18a126be-89fc-4b44-9302-0c8c32564a5c None None] MFA Rules not processed for user `1854af616f344b2c9cb6ab4e5c022ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:10.003081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-18a126be-89fc-4b44-9302-0c8c32564a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:10.068958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1254/2541] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:10.073742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dae9a-6d5d-43f9-a17f-166adb48415b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:10.150440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dae9a-6d5d-43f9-a17f-166adb48415b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:10.156418 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0734c-fdc2-4a1a-b99b-0ed80640515b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:10.157125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0734c-fdc2-4a1a-b99b-0ed80640515b admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:10.157406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0734c-fdc2-4a1a-b99b-0ed80640515b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:10.157689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0734c-fdc2-4a1a-b99b-0ed80640515b admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:10.158376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0734c-fdc2-4a1a-b99b-0ed80640515b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:10.160358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0734c-fdc2-4a1a-b99b-0ed80640515b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:10.189047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1288/2542] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:10 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:10.193363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a126be-89fc-4b44-9302-0c8c32564a5c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:10.204845 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a126be-89fc-4b44-9302-0c8c32564a5c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:10.210670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:10.265280 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65eb1a-0b1a-4f5d-a929-7aad1b13037b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:10.265958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65eb1a-0b1a-4f5d-a929-7aad1b13037b admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:10.266241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65eb1a-0b1a-4f5d-a929-7aad1b13037b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:10.266513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65eb1a-0b1a-4f5d-a929-7aad1b13037b admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:10.267260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65eb1a-0b1a-4f5d-a929-7aad1b13037b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:10.269207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65eb1a-0b1a-4f5d-a929-7aad1b13037b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:10.292210 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1255/2543] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:10 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:10.296382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0734c-fdc2-4a1a-b99b-0ed80640515b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:10.307362 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0734c-fdc2-4a1a-b99b-0ed80640515b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:10.313081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294a04e-97f5-4608-b5bf-419ea62b4cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:10.313730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294a04e-97f5-4608-b5bf-419ea62b4cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:10.314024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294a04e-97f5-4608-b5bf-419ea62b4cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:10.314290 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294a04e-97f5-4608-b5bf-419ea62b4cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:10.315056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294a04e-97f5-4608-b5bf-419ea62b4cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:10.316683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294a04e-97f5-4608-b5bf-419ea62b4cd6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:10.327667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1289/2544] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:10.331852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65eb1a-0b1a-4f5d-a929-7aad1b13037b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:10.345271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65eb1a-0b1a-4f5d-a929-7aad1b13037b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:10.352226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7887b0-2d9b-477a-90d4-1843022d70ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:10.353567 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7887b0-2d9b-477a-90d4-1843022d70ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:10.354039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7887b0-2d9b-477a-90d4-1843022d70ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:10.354320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7887b0-2d9b-477a-90d4-1843022d70ed admin admin] PATH_INFO: `/v3/projects/e994c654361148cebb27735db93c59d9/users/3ffb9c439aa744d087b1ff75e4187245/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:10.371069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7887b0-2d9b-477a-90d4-1843022d70ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=e994c654361148cebb27735db93c59d9, user_id=3ffb9c439aa744d087b1ff75e4187245, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:10.373738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7887b0-2d9b-477a-90d4-1843022d70ed admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:10.389748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1256/2545] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:10 2022] PUT /identity/v3/projects/e994c654361148cebb27735db93c59d9/users/3ffb9c439aa744d087b1ff75e4187245/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:10.393843 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294a04e-97f5-4608-b5bf-419ea62b4cd6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:10.404548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294a04e-97f5-4608-b5bf-419ea62b4cd6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:10.464871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481c14a-6e9b-4448-a818-d382d945ba9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:10.465621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481c14a-6e9b-4448-a818-d382d945ba9b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:10.465901 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481c14a-6e9b-4448-a818-d382d945ba9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:10.466170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481c14a-6e9b-4448-a818-d382d945ba9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:10.466933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481c14a-6e9b-4448-a818-d382d945ba9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:10.468542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481c14a-6e9b-4448-a818-d382d945ba9b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:10.478987 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1290/2546] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:10.483266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7887b0-2d9b-477a-90d4-1843022d70ed admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:10.494392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7887b0-2d9b-477a-90d4-1843022d70ed admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:10.499430 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a88aa-73f7-475a-b428-77ee02684f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:10.500335 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a88aa-73f7-475a-b428-77ee02684f8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:10.500617 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a88aa-73f7-475a-b428-77ee02684f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:10.500880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a88aa-73f7-475a-b428-77ee02684f8a admin admin] PATH_INFO: `/v3/projects/e994c654361148cebb27735db93c59d9/users/3ffb9c439aa744d087b1ff75e4187245/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:10.502930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a88aa-73f7-475a-b428-77ee02684f8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e994c654361148cebb27735db93c59d9, user_id=3ffb9c439aa744d087b1ff75e4187245, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:10.506512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a88aa-73f7-475a-b428-77ee02684f8a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:10.517571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1257/2547] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:10 2022] PUT /identity/v3/projects/e994c654361148cebb27735db93c59d9/users/3ffb9c439aa744d087b1ff75e4187245/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:10.528415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b3602-0363-4517-a963-1d4447779130 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:10.528779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b3602-0363-4517-a963-1d4447779130 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:10.529122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b3602-0363-4517-a963-1d4447779130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:10.556075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-596b3602-0363-4517-a963-1d4447779130 None None] MFA Rules not processed for user `3ffb9c439aa744d087b1ff75e4187245`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:10.557284 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-596b3602-0363-4517-a963-1d4447779130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:10.565802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:10.631768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1291/2548] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:10.643611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a88aa-73f7-475a-b428-77ee02684f8a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:10.645653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b36a88aa-73f7-475a-b428-77ee02684f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:10.711086 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a88aa-73f7-475a-b428-77ee02684f8a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:10.716050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288697d-22c4-4b7d-8941-8d5b3cbb26a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:10.716694 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288697d-22c4-4b7d-8941-8d5b3cbb26a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:10.716976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288697d-22c4-4b7d-8941-8d5b3cbb26a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:10.717249 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288697d-22c4-4b7d-8941-8d5b3cbb26a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:10.727483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288697d-22c4-4b7d-8941-8d5b3cbb26a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:10.728927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288697d-22c4-4b7d-8941-8d5b3cbb26a3 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:10.738385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1258/2549] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:10.815391 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b3602-0363-4517-a963-1d4447779130 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:10.825782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b3602-0363-4517-a963-1d4447779130 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:10.829584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53460735-ccbc-4dc5-80d5-675ec9c9ec65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:10.830254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53460735-ccbc-4dc5-80d5-675ec9c9ec65 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:10.830566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53460735-ccbc-4dc5-80d5-675ec9c9ec65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:10.830883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53460735-ccbc-4dc5-80d5-675ec9c9ec65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:10.897650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53460735-ccbc-4dc5-80d5-675ec9c9ec65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:10.899174 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53460735-ccbc-4dc5-80d5-675ec9c9ec65 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:10.908403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1292/2550] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:20.734712 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=66,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:20.904179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=53 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:22.005648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288697d-22c4-4b7d-8941-8d5b3cbb26a3 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:22.077114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288697d-22c4-4b7d-8941-8d5b3cbb26a3 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:22.082864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:22.083815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:22.084098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:22.084376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] PATH_INFO: `/v3/users/1854af616f344b2c9cb6ab4e5c022ed4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:22.085735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1854af616f344b2c9cb6ab4e5c022ed4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:22.087715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:22.141860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1854af616f344b2c9cb6ab4e5c022ed4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.142508 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1854af616f344b2c9cb6ab4e5c022ed4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.154428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1854af616f344b2c9cb6ab4e5c022ed4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.155104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1854af616f344b2c9cb6ab4e5c022ed4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.167251 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1854af616f344b2c9cb6ab4e5c022ed4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.181002 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1259/2551] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:22 2022] DELETE /identity/v3/users/1854af616f344b2c9cb6ab4e5c022ed4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:22.185623 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5e925-e1bf-4d26-bdb5-5bd3f4822dd0 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:22.186088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5e925-e1bf-4d26-bdb5-5bd3f4822dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:22.186243 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5e925-e1bf-4d26-bdb5-5bd3f4822dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:22.204512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f6e5e925-e1bf-4d26-bdb5-5bd3f4822dd0 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:22.205696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f6e5e925-e1bf-4d26-bdb5-5bd3f4822dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:22.265359 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1293/2552] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:22.275304 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:22.276703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:22.340974 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e74ef-cfd6-4c1f-804c-3b386b86463c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:22.344922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d1f19-3bf4-4289-ab4a-3870c66ab171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:22.345582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d1f19-3bf4-4289-ab4a-3870c66ab171 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:22.345867 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d1f19-3bf4-4289-ab4a-3870c66ab171 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:22.346147 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d1f19-3bf4-4289-ab4a-3870c66ab171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:22.356462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d1f19-3bf4-4289-ab4a-3870c66ab171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:22.358057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d1f19-3bf4-4289-ab4a-3870c66ab171 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:22.367804 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1260/2553] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:22.687683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5e925-e1bf-4d26-bdb5-5bd3f4822dd0 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:22.689191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f6e5e925-e1bf-4d26-bdb5-5bd3f4822dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:22.701592 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5e925-e1bf-4d26-bdb5-5bd3f4822dd0 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:22.707544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:22.708468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:22.708732 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:22.709081 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] PATH_INFO: `/v3/projects/c0780588ce2d4e73a651a310891c9322` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:22.710370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0780588ce2d4e73a651a310891c9322)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:22.713179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:22.769547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0780588ce2d4e73a651a310891c9322', 'request_id': 'req-a91bdfda-a357-4c87-8fad-f151015a2ca9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.777118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0780588ce2d4e73a651a310891c9322', 'request_id': 'req-a91bdfda-a357-4c87-8fad-f151015a2ca9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.782881 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0780588ce2d4e73a651a310891c9322', 'request_id': 'req-a91bdfda-a357-4c87-8fad-f151015a2ca9'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.796796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] The token cache is being invalidate because project c0780588ce2d4e73a651a310891c9322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:22.797085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.797697 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.799292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1294/2554] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:22 2022] DELETE /identity/v3/projects/c0780588ce2d4e73a651a310891c9322 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:22.803718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d1f19-3bf4-4289-ab4a-3870c66ab171 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:22.813530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d1f19-3bf4-4289-ab4a-3870c66ab171 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:22.833157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:22.877463 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c301c-f773-44d6-a1cc-b221adbda9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:22.878326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c301c-f773-44d6-a1cc-b221adbda9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:22.878608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c301c-f773-44d6-a1cc-b221adbda9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:22.878920 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c301c-f773-44d6-a1cc-b221adbda9f1 admin admin] PATH_INFO: `/v3/users/3ffb9c439aa744d087b1ff75e4187245` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:22.880185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c301c-f773-44d6-a1cc-b221adbda9f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ffb9c439aa744d087b1ff75e4187245)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:22.882152 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c301c-f773-44d6-a1cc-b221adbda9f1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:22.952441 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2d1c301c-f773-44d6-a1cc-b221adbda9f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ffb9c439aa744d087b1ff75e4187245'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.953078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2d1c301c-f773-44d6-a1cc-b221adbda9f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ffb9c439aa744d087b1ff75e4187245'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.967128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2d1c301c-f773-44d6-a1cc-b221adbda9f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ffb9c439aa744d087b1ff75e4187245'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.967921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2d1c301c-f773-44d6-a1cc-b221adbda9f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ffb9c439aa744d087b1ff75e4187245'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.977346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2d1c301c-f773-44d6-a1cc-b221adbda9f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ffb9c439aa744d087b1ff75e4187245'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:22.996164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1261/2555] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:22 2022] DELETE /identity/v3/users/3ffb9c439aa744d087b1ff75e4187245 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:23.297180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:23.298791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:23.328912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bdfda-a357-4c87-8fad-f151015a2ca9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:23.391610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:23.392649 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:23.393037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:23.393380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] PATH_INFO: `/v3/projects/e994c654361148cebb27735db93c59d9` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:23.394760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e994c654361148cebb27735db93c59d9)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:23.396810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:23.432226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:23.456767 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e994c654361148cebb27735db93c59d9', 'request_id': 'req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:23.469242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e994c654361148cebb27735db93c59d9', 'request_id': 'req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:23.475282 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e994c654361148cebb27735db93c59d9', 'request_id': 'req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:23.489243 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] The token cache is being invalidate because project e994c654361148cebb27735db93c59d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:23.489533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:23.490148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-498a7601-d33a-4cb2-adb4-f4f9b51e8e33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:23.491766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1295/2556] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:23 2022] DELETE /identity/v3/projects/e994c654361148cebb27735db93c59d9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:23.497003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c297be-ebe5-4cb5-afac-5e00e2f41838 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:23.497515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c297be-ebe5-4cb5-afac-5e00e2f41838 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:23.497619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c297be-ebe5-4cb5-afac-5e00e2f41838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:23.516081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-34c297be-ebe5-4cb5-afac-5e00e2f41838 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:23.517281 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-34c297be-ebe5-4cb5-afac-5e00e2f41838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:23.576604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1262/2557] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:23.586235 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ac560-265b-47f2-abce-1c3f8485ad7a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:23.586630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ac560-265b-47f2-abce-1c3f8485ad7a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:23.586945 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ac560-265b-47f2-abce-1c3f8485ad7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:23.608205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a30ac560-265b-47f2-abce-1c3f8485ad7a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:23.609443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a30ac560-265b-47f2-abce-1c3f8485ad7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:23.618024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1296/2558] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:23.624306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c297be-ebe5-4cb5-afac-5e00e2f41838 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:23.639346 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c297be-ebe5-4cb5-afac-5e00e2f41838 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:23.648205 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514430fc-4138-4da5-b134-b764bb89028b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:23.649123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514430fc-4138-4da5-b134-b764bb89028b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:23.649485 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514430fc-4138-4da5-b134-b764bb89028b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:23.649824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514430fc-4138-4da5-b134-b764bb89028b admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:23.650656 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514430fc-4138-4da5-b134-b764bb89028b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:23.650998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514430fc-4138-4da5-b134-b764bb89028b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:23.652641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514430fc-4138-4da5-b134-b764bb89028b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:23.664222 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1263/2559] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:49:23.668583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ac560-265b-47f2-abce-1c3f8485ad7a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:23.677664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ac560-265b-47f2-abce-1c3f8485ad7a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:23.684002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed454fd9-95fd-4dd2-a328-96f49f54d6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:23.684652 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed454fd9-95fd-4dd2-a328-96f49f54d6db admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:23.684933 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed454fd9-95fd-4dd2-a328-96f49f54d6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:23.685199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed454fd9-95fd-4dd2-a328-96f49f54d6db admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:23.685900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed454fd9-95fd-4dd2-a328-96f49f54d6db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:23.687857 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed454fd9-95fd-4dd2-a328-96f49f54d6db admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:23.722452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1297/2560] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:23 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:23.727084 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514430fc-4138-4da5-b134-b764bb89028b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:23.745728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514430fc-4138-4da5-b134-b764bb89028b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:23.787636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:23.834499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356f3f4-00e4-43e3-bfd2-dd7d9b652ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:23.837564 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356f3f4-00e4-43e3-bfd2-dd7d9b652ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:23.839318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356f3f4-00e4-43e3-bfd2-dd7d9b652ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:23.839683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356f3f4-00e4-43e3-bfd2-dd7d9b652ee1 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:23.840649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356f3f4-00e4-43e3-bfd2-dd7d9b652ee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:23.843077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356f3f4-00e4-43e3-bfd2-dd7d9b652ee1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:23.870272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1264/2561] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:23 2022] POST /identity/v3/users => generated 283 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:23.874240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed454fd9-95fd-4dd2-a328-96f49f54d6db admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:23.886237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed454fd9-95fd-4dd2-a328-96f49f54d6db admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:23.891632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3c6c5-691c-44c7-80d9-938704097b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:23.892319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3c6c5-691c-44c7-80d9-938704097b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:23.892603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3c6c5-691c-44c7-80d9-938704097b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:23.893032 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3c6c5-691c-44c7-80d9-938704097b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:23.893777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3c6c5-691c-44c7-80d9-938704097b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:23.903036 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3c6c5-691c-44c7-80d9-938704097b73 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:23.919851 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1298/2562] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:23.920919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356f3f4-00e4-43e3-bfd2-dd7d9b652ee1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:23.932049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356f3f4-00e4-43e3-bfd2-dd7d9b652ee1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:23.936962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1a3f7-81ae-449d-acf7-0e71ff8b4f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:23.937638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a3f7-81ae-449d-acf7-0e71ff8b4f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:23.937922 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a3f7-81ae-449d-acf7-0e71ff8b4f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:23.938196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1a3f7-81ae-449d-acf7-0e71ff8b4f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:23.938958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1a3f7-81ae-449d-acf7-0e71ff8b4f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:23.940614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1a3f7-81ae-449d-acf7-0e71ff8b4f1d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:23.951513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1265/2563] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:23.955740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3c6c5-691c-44c7-80d9-938704097b73 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:23.966831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3c6c5-691c-44c7-80d9-938704097b73 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:23.971745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe7b3e-255b-4a48-a045-55d1bb5b1507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:23.972862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe7b3e-255b-4a48-a045-55d1bb5b1507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:23.973382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe7b3e-255b-4a48-a045-55d1bb5b1507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:23.973878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe7b3e-255b-4a48-a045-55d1bb5b1507 admin admin] PATH_INFO: `/v3/projects/e9b747fb6a4246bf949404b624f50ec0/users/2125ab5feb514ea1a1dad2e4e940f0bd/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:23.990895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe7b3e-255b-4a48-a045-55d1bb5b1507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9b747fb6a4246bf949404b624f50ec0, user_id=2125ab5feb514ea1a1dad2e4e940f0bd, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:23.993716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe7b3e-255b-4a48-a045-55d1bb5b1507 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:24.018785 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1299/2564] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:23 2022] PUT /identity/v3/projects/e9b747fb6a4246bf949404b624f50ec0/users/2125ab5feb514ea1a1dad2e4e940f0bd/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:24.021113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0156ce0-114c-4be8-8d6e-5875933d4e73 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:24.021410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0156ce0-114c-4be8-8d6e-5875933d4e73 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:24.021690 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0156ce0-114c-4be8-8d6e-5875933d4e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:24.051432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c0156ce0-114c-4be8-8d6e-5875933d4e73 None None] MFA Rules not processed for user `2125ab5feb514ea1a1dad2e4e940f0bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:24.052778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c0156ce0-114c-4be8-8d6e-5875933d4e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:24.137005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1266/2565] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:24.141950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe7b3e-255b-4a48-a045-55d1bb5b1507 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:24.217516 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe7b3e-255b-4a48-a045-55d1bb5b1507 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:24.229413 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86ae4c-e104-455e-80f1-80a0d585454f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:24.230116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86ae4c-e104-455e-80f1-80a0d585454f admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:24.230415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86ae4c-e104-455e-80f1-80a0d585454f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:24.230716 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86ae4c-e104-455e-80f1-80a0d585454f admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:24.231493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86ae4c-e104-455e-80f1-80a0d585454f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:24.233419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a86ae4c-e104-455e-80f1-80a0d585454f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:24.264996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1300/2566] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:24 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:24.266233 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0156ce0-114c-4be8-8d6e-5875933d4e73 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:24.277439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0156ce0-114c-4be8-8d6e-5875933d4e73 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:24.336495 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db6dc3-706f-4579-aa20-6608a8ed1c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:24.337168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db6dc3-706f-4579-aa20-6608a8ed1c53 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:24.337737 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db6dc3-706f-4579-aa20-6608a8ed1c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:24.338425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db6dc3-706f-4579-aa20-6608a8ed1c53 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:24.339510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db6dc3-706f-4579-aa20-6608a8ed1c53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:24.341450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db6dc3-706f-4579-aa20-6608a8ed1c53 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:24.367103 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1267/2567] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:24 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:24.368396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86ae4c-e104-455e-80f1-80a0d585454f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:24.379336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a86ae4c-e104-455e-80f1-80a0d585454f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:24.385123 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec56819-32aa-4af4-b9a6-7c241129dbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:24.385794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec56819-32aa-4af4-b9a6-7c241129dbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:24.386076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec56819-32aa-4af4-b9a6-7c241129dbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:24.386352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec56819-32aa-4af4-b9a6-7c241129dbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:24.387132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec56819-32aa-4af4-b9a6-7c241129dbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:24.388721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec56819-32aa-4af4-b9a6-7c241129dbe3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:24.403620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1301/2568] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:24.404911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db6dc3-706f-4579-aa20-6608a8ed1c53 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:24.415575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db6dc3-706f-4579-aa20-6608a8ed1c53 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:24.420755 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbf7b1-2f31-44be-a1b3-8ccb7ab07e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:24.421741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbf7b1-2f31-44be-a1b3-8ccb7ab07e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:24.422011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbf7b1-2f31-44be-a1b3-8ccb7ab07e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:24.422637 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbf7b1-2f31-44be-a1b3-8ccb7ab07e49 admin admin] PATH_INFO: `/v3/projects/9d6fa73910364585adfcc6d46294b255/users/cf8f49f0a537414d892c2ef000ae8c5c/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:24.440914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbf7b1-2f31-44be-a1b3-8ccb7ab07e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6fa73910364585adfcc6d46294b255, user_id=cf8f49f0a537414d892c2ef000ae8c5c, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:24.443792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbf7b1-2f31-44be-a1b3-8ccb7ab07e49 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:24.463218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1268/2569] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:24 2022] PUT /identity/v3/projects/9d6fa73910364585adfcc6d46294b255/users/cf8f49f0a537414d892c2ef000ae8c5c/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:24.467271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec56819-32aa-4af4-b9a6-7c241129dbe3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:24.477415 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec56819-32aa-4af4-b9a6-7c241129dbe3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:24.499620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:24.544330 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4cc62-2860-46d2-aaea-23ec6043a74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:24.545171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4cc62-2860-46d2-aaea-23ec6043a74d admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:24.545523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4cc62-2860-46d2-aaea-23ec6043a74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:24.545779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4cc62-2860-46d2-aaea-23ec6043a74d admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:24.547189 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4cc62-2860-46d2-aaea-23ec6043a74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:24.549522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4cc62-2860-46d2-aaea-23ec6043a74d admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:24.564175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1302/2570] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:24.568531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbf7b1-2f31-44be-a1b3-8ccb7ab07e49 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:24.575274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:24.580854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbf7b1-2f31-44be-a1b3-8ccb7ab07e49 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:24.586754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a23de0-4b5f-4a66-bd32-6daec6593bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:24.588392 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a23de0-4b5f-4a66-bd32-6daec6593bff admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:24.588681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a23de0-4b5f-4a66-bd32-6daec6593bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:24.589311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a23de0-4b5f-4a66-bd32-6daec6593bff admin admin] PATH_INFO: `/v3/projects/9d6fa73910364585adfcc6d46294b255/users/cf8f49f0a537414d892c2ef000ae8c5c/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:24.591385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a23de0-4b5f-4a66-bd32-6daec6593bff admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d6fa73910364585adfcc6d46294b255, user_id=cf8f49f0a537414d892c2ef000ae8c5c, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:24.594036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a23de0-4b5f-4a66-bd32-6daec6593bff admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:24.605524 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1269/2571] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:24 2022] PUT /identity/v3/projects/9d6fa73910364585adfcc6d46294b255/users/cf8f49f0a537414d892c2ef000ae8c5c/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:24.615702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4fd42-fe48-478d-8a67-b7333ed60774 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:24.616011 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4fd42-fe48-478d-8a67-b7333ed60774 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:24.616288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab4fd42-fe48-478d-8a67-b7333ed60774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:24.643707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fab4fd42-fe48-478d-8a67-b7333ed60774 None None] MFA Rules not processed for user `cf8f49f0a537414d892c2ef000ae8c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:24.645086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fab4fd42-fe48-478d-8a67-b7333ed60774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:24.721087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1303/2572] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:24 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:24.733897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a23de0-4b5f-4a66-bd32-6daec6593bff admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:24.735936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-22a23de0-4b5f-4a66-bd32-6daec6593bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:24.797329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a23de0-4b5f-4a66-bd32-6daec6593bff admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:24.801247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b85f5-2ac9-4bca-b54d-5a394742cd04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:24.801902 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b85f5-2ac9-4bca-b54d-5a394742cd04 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:24.802183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b85f5-2ac9-4bca-b54d-5a394742cd04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:24.802462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b85f5-2ac9-4bca-b54d-5a394742cd04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:24.812568 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b85f5-2ac9-4bca-b54d-5a394742cd04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:24.814285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b85f5-2ac9-4bca-b54d-5a394742cd04 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:24.824375 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1270/2573] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:24 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:24.899488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4fd42-fe48-478d-8a67-b7333ed60774 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:24.909735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab4fd42-fe48-478d-8a67-b7333ed60774 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:24.913746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe9436-c06e-42de-b5ec-d6a0db400578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:24.914395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe9436-c06e-42de-b5ec-d6a0db400578 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:24.914678 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe9436-c06e-42de-b5ec-d6a0db400578 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:24.914985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe9436-c06e-42de-b5ec-d6a0db400578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:24.972972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe9436-c06e-42de-b5ec-d6a0db400578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:24.974407 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe9436-c06e-42de-b5ec-d6a0db400578 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:24.984640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1304/2574] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:25.534722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b85f5-2ac9-4bca-b54d-5a394742cd04 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:25.536421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ce0b85f5-2ac9-4bca-b54d-5a394742cd04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:25.605937 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b85f5-2ac9-4bca-b54d-5a394742cd04 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:25.611044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:25.611931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:25.612204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:25.612479 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] PATH_INFO: `/v3/users/f5a46d3150ec496ebe7342def072e7fc` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:25.613723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a46d3150ec496ebe7342def072e7fc)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:25.615683 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:25.665649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a46d3150ec496ebe7342def072e7fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:25.666365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a46d3150ec496ebe7342def072e7fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:25.679453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a46d3150ec496ebe7342def072e7fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:25.680237 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a46d3150ec496ebe7342def072e7fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:25.691185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a46d3150ec496ebe7342def072e7fc'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:25.704772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1271/2575] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:25 2022] DELETE /identity/v3/users/f5a46d3150ec496ebe7342def072e7fc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:25.709472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f3b0-c889-4979-9a8b-36d39a9f5f62 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:25.709991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f3b0-c889-4979-9a8b-36d39a9f5f62 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:25.709991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f3b0-c889-4979-9a8b-36d39a9f5f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:25.729970 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-09b2f3b0-c889-4979-9a8b-36d39a9f5f62 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:25.731217 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-09b2f3b0-c889-4979-9a8b-36d39a9f5f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:25.771795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:25.801746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1305/2576] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:25.812095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:25.813808 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:25.828608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:25.876462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87927f-9cdb-469e-b5cb-dffc85ed4620 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:25.880360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d068b-ed7f-43e7-a359-5013d2b95679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:25.881010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d068b-ed7f-43e7-a359-5013d2b95679 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:25.881286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d068b-ed7f-43e7-a359-5013d2b95679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:25.881558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d068b-ed7f-43e7-a359-5013d2b95679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:25.891005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d068b-ed7f-43e7-a359-5013d2b95679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:25.892429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d068b-ed7f-43e7-a359-5013d2b95679 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:25.902266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1272/2577] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:26.179180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f3b0-c889-4979-9a8b-36d39a9f5f62 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:26.180753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-09b2f3b0-c889-4979-9a8b-36d39a9f5f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:26.204248 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f3b0-c889-4979-9a8b-36d39a9f5f62 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:26.211737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:26.212583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:26.212976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:26.213209 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] PATH_INFO: `/v3/projects/f6c9fe1eae84418680169a11a739ab60` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:26.215117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c9fe1eae84418680169a11a739ab60)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:26.217092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:26.277793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c9fe1eae84418680169a11a739ab60', 'request_id': 'req-0c231718-b31a-4d37-a25a-19931c70de15'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.284130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c9fe1eae84418680169a11a739ab60', 'request_id': 'req-0c231718-b31a-4d37-a25a-19931c70de15'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.289584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c9fe1eae84418680169a11a739ab60', 'request_id': 'req-0c231718-b31a-4d37-a25a-19931c70de15'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.304345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] The token cache is being invalidate because project f6c9fe1eae84418680169a11a739ab60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:26.304709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.305427 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.307396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1306/2578] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:26 2022] DELETE /identity/v3/projects/f6c9fe1eae84418680169a11a739ab60 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:26.311471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d068b-ed7f-43e7-a359-5013d2b95679 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:26.321645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d068b-ed7f-43e7-a359-5013d2b95679 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:26.381036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c1ac99-8319-4a67-86e5-95bbb778b1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:26.381919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ac99-8319-4a67-86e5-95bbb778b1f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:26.382198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ac99-8319-4a67-86e5-95bbb778b1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:26.382470 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c1ac99-8319-4a67-86e5-95bbb778b1f9 admin admin] PATH_INFO: `/v3/users/cd8b8c2e03ea4ed28599a71ba4c50068` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:26.383698 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1ac99-8319-4a67-86e5-95bbb778b1f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd8b8c2e03ea4ed28599a71ba4c50068)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:26.385649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c1ac99-8319-4a67-86e5-95bbb778b1f9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:26.433423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b0c1ac99-8319-4a67-86e5-95bbb778b1f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd8b8c2e03ea4ed28599a71ba4c50068'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.434328 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b0c1ac99-8319-4a67-86e5-95bbb778b1f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd8b8c2e03ea4ed28599a71ba4c50068'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.446381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b0c1ac99-8319-4a67-86e5-95bbb778b1f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd8b8c2e03ea4ed28599a71ba4c50068'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.447111 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b0c1ac99-8319-4a67-86e5-95bbb778b1f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd8b8c2e03ea4ed28599a71ba4c50068'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.458014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-b0c1ac99-8319-4a67-86e5-95bbb778b1f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd8b8c2e03ea4ed28599a71ba4c50068'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.471853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1273/2579] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:26 2022] DELETE /identity/v3/users/cd8b8c2e03ea4ed28599a71ba4c50068 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:26.752027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:26.753480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:26.769600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c231718-b31a-4d37-a25a-19931c70de15 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:26.834086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:26.835132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:26.835446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:26.835725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] PATH_INFO: `/v3/projects/2a4363bad0e341868e161e187cc99041` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:26.836937 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4363bad0e341868e161e187cc99041)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:26.839342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:26.890162 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4363bad0e341868e161e187cc99041', 'request_id': 'req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.896370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4363bad0e341868e161e187cc99041', 'request_id': 'req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.901893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4363bad0e341868e161e187cc99041', 'request_id': 'req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.911843 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:26.918333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] The token cache is being invalidate because project 2a4363bad0e341868e161e187cc99041 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:26.918794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.919765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e00df5e2-d6ed-4e17-86b5-8353cb2c7c2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:26.922508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1307/2580] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:26 2022] DELETE /identity/v3/projects/2a4363bad0e341868e161e187cc99041 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:26.928877 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efad37-dd9d-470c-807b-888a2f15ca45 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:26.929369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efad37-dd9d-470c-807b-888a2f15ca45 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:26.929496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0efad37-dd9d-470c-807b-888a2f15ca45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:26.949527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b0efad37-dd9d-470c-807b-888a2f15ca45 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:26.950735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b0efad37-dd9d-470c-807b-888a2f15ca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:26.982157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:27.024270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1274/2581] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:27.033340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e41a70-9e4d-4daa-a0d6-6b60b73870bb None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.033629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e41a70-9e4d-4daa-a0d6-6b60b73870bb None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.033901 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e41a70-9e4d-4daa-a0d6-6b60b73870bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.053295 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-87e41a70-9e4d-4daa-a0d6-6b60b73870bb None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:27.054539 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-87e41a70-9e4d-4daa-a0d6-6b60b73870bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:27.063086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1308/2582] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:27.068006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0efad37-dd9d-470c-807b-888a2f15ca45 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.083977 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0efad37-dd9d-470c-807b-888a2f15ca45 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.091624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b8550-8ad0-492c-85ed-2d53a69eb2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.092285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b8550-8ad0-492c-85ed-2d53a69eb2eb admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.092566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b8550-8ad0-492c-85ed-2d53a69eb2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.092834 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b8550-8ad0-492c-85ed-2d53a69eb2eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.093563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b8550-8ad0-492c-85ed-2d53a69eb2eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:27.093864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b8550-8ad0-492c-85ed-2d53a69eb2eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.095516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b8550-8ad0-492c-85ed-2d53a69eb2eb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.107983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1275/2583] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:49:27.113919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e41a70-9e4d-4daa-a0d6-6b60b73870bb None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.124031 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e41a70-9e4d-4daa-a0d6-6b60b73870bb None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.129806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae27784-bbd9-45fc-8e42-70b85c8dfc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.130463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae27784-bbd9-45fc-8e42-70b85c8dfc16 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.130849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae27784-bbd9-45fc-8e42-70b85c8dfc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.131122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae27784-bbd9-45fc-8e42-70b85c8dfc16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.131827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae27784-bbd9-45fc-8e42-70b85c8dfc16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.133770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae27784-bbd9-45fc-8e42-70b85c8dfc16 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.163675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1309/2584] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:27 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:27.168571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b8550-8ad0-492c-85ed-2d53a69eb2eb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.179356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b8550-8ad0-492c-85ed-2d53a69eb2eb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.241134 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af718761-b5d8-4517-a21b-f16ca3508c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.241813 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718761-b5d8-4517-a21b-f16ca3508c16 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.242096 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718761-b5d8-4517-a21b-f16ca3508c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.242364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af718761-b5d8-4517-a21b-f16ca3508c16 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.243080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af718761-b5d8-4517-a21b-f16ca3508c16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.245083 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af718761-b5d8-4517-a21b-f16ca3508c16 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.271182 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1276/2585] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:27 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:27.275948 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae27784-bbd9-45fc-8e42-70b85c8dfc16 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.286535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae27784-bbd9-45fc-8e42-70b85c8dfc16 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.291685 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a66515-59eb-419d-9708-02586b7231f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.292346 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66515-59eb-419d-9708-02586b7231f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.292630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66515-59eb-419d-9708-02586b7231f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.292905 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66515-59eb-419d-9708-02586b7231f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.293641 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a66515-59eb-419d-9708-02586b7231f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.295288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a66515-59eb-419d-9708-02586b7231f1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.305623 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1310/2586] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:27.309948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af718761-b5d8-4517-a21b-f16ca3508c16 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.320516 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af718761-b5d8-4517-a21b-f16ca3508c16 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.325728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ade15-4fa7-44d2-8edf-ab5d2570ab5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.326391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ade15-4fa7-44d2-8edf-ab5d2570ab5b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.326716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ade15-4fa7-44d2-8edf-ab5d2570ab5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.327008 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ade15-4fa7-44d2-8edf-ab5d2570ab5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.327744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ade15-4fa7-44d2-8edf-ab5d2570ab5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.329359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ade15-4fa7-44d2-8edf-ab5d2570ab5b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.339872 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1277/2587] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:27.343869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a66515-59eb-419d-9708-02586b7231f1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.354047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a66515-59eb-419d-9708-02586b7231f1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.359115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a65e8e-4a2f-48e3-a658-e8569be2ee36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.360001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a65e8e-4a2f-48e3-a658-e8569be2ee36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.360281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a65e8e-4a2f-48e3-a658-e8569be2ee36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.360552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a65e8e-4a2f-48e3-a658-e8569be2ee36 admin admin] PATH_INFO: `/v3/projects/0742f9481a2d41228f2f72fd62ecbf82/users/9771c1b7166541bcbea0c3dee75cd339/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.375582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a65e8e-4a2f-48e3-a658-e8569be2ee36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0742f9481a2d41228f2f72fd62ecbf82, user_id=9771c1b7166541bcbea0c3dee75cd339, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.378231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a65e8e-4a2f-48e3-a658-e8569be2ee36 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.394160 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1311/2588] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:27 2022] PUT /identity/v3/projects/0742f9481a2d41228f2f72fd62ecbf82/users/9771c1b7166541bcbea0c3dee75cd339/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:27.404767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8f21c-e400-4b60-aa8c-c8203b1c647d None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.405061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8f21c-e400-4b60-aa8c-c8203b1c647d None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.405344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8f21c-e400-4b60-aa8c-c8203b1c647d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.432970 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7bd8f21c-e400-4b60-aa8c-c8203b1c647d None None] MFA Rules not processed for user `9771c1b7166541bcbea0c3dee75cd339`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:27.434174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7bd8f21c-e400-4b60-aa8c-c8203b1c647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:27.510960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1278/2589] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:27.518360 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a65e8e-4a2f-48e3-a658-e8569be2ee36 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.596211 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a65e8e-4a2f-48e3-a658-e8569be2ee36 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.602230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39821680-79e8-4f64-bc81-ee031e97c226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.602902 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39821680-79e8-4f64-bc81-ee031e97c226 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.603186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39821680-79e8-4f64-bc81-ee031e97c226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.603521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39821680-79e8-4f64-bc81-ee031e97c226 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.604304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39821680-79e8-4f64-bc81-ee031e97c226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.606191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39821680-79e8-4f64-bc81-ee031e97c226 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.634156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1312/2590] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:27 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:27.638316 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8f21c-e400-4b60-aa8c-c8203b1c647d None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.650789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8f21c-e400-4b60-aa8c-c8203b1c647d None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.682071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:27.716220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afa18f-f845-460b-b6f4-26d4aa8c4fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.716914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa18f-f845-460b-b6f4-26d4aa8c4fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.717202 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa18f-f845-460b-b6f4-26d4aa8c4fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.717483 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa18f-f845-460b-b6f4-26d4aa8c4fe7 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.718189 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afa18f-f845-460b-b6f4-26d4aa8c4fe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.720192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6afa18f-f845-460b-b6f4-26d4aa8c4fe7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.729024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39821680-79e8-4f64-bc81-ee031e97c226 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.730988 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-39821680-79e8-4f64-bc81-ee031e97c226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:27.745034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39821680-79e8-4f64-bc81-ee031e97c226 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.747217 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1279/2591] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:27 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:27.749986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894118a5-a189-413d-85fd-bd7e1805af3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.751008 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afa18f-f845-460b-b6f4-26d4aa8c4fe7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.751151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894118a5-a189-413d-85fd-bd7e1805af3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.751347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894118a5-a189-413d-85fd-bd7e1805af3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.751752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894118a5-a189-413d-85fd-bd7e1805af3f admin admin] PATH_INFO: `/v3/users/2125ab5feb514ea1a1dad2e4e940f0bd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.752963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894118a5-a189-413d-85fd-bd7e1805af3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2125ab5feb514ea1a1dad2e4e940f0bd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.754924 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894118a5-a189-413d-85fd-bd7e1805af3f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.761090 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afa18f-f845-460b-b6f4-26d4aa8c4fe7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.765775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81373ce6-7dfb-405a-8e7f-5e1e743bdd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.766515 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81373ce6-7dfb-405a-8e7f-5e1e743bdd25 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.766896 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81373ce6-7dfb-405a-8e7f-5e1e743bdd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.767179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81373ce6-7dfb-405a-8e7f-5e1e743bdd25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.767909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81373ce6-7dfb-405a-8e7f-5e1e743bdd25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.769763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81373ce6-7dfb-405a-8e7f-5e1e743bdd25 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.779964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1280/2592] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:27.784066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81373ce6-7dfb-405a-8e7f-5e1e743bdd25 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.794487 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81373ce6-7dfb-405a-8e7f-5e1e743bdd25 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.799438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1d1a4-5fa2-4e45-935f-fd19aad4d856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.800324 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1d1a4-5fa2-4e45-935f-fd19aad4d856 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.800607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1d1a4-5fa2-4e45-935f-fd19aad4d856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.800887 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1d1a4-5fa2-4e45-935f-fd19aad4d856 admin admin] PATH_INFO: `/v3/projects/dd6904f93698498c9e72af4a110ce16a/users/6807a8c7d5e04dd9adb155bd2dad4c2b/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.805756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-894118a5-a189-413d-85fd-bd7e1805af3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2125ab5feb514ea1a1dad2e4e940f0bd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:27.806548 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-894118a5-a189-413d-85fd-bd7e1805af3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2125ab5feb514ea1a1dad2e4e940f0bd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:27.816208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1d1a4-5fa2-4e45-935f-fd19aad4d856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6904f93698498c9e72af4a110ce16a, user_id=6807a8c7d5e04dd9adb155bd2dad4c2b, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.818895 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1d1a4-5fa2-4e45-935f-fd19aad4d856 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.819230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-894118a5-a189-413d-85fd-bd7e1805af3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2125ab5feb514ea1a1dad2e4e940f0bd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:27.819943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-894118a5-a189-413d-85fd-bd7e1805af3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2125ab5feb514ea1a1dad2e4e940f0bd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:27.829621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-894118a5-a189-413d-85fd-bd7e1805af3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2125ab5feb514ea1a1dad2e4e940f0bd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:27.831740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:27.834499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1281/2593] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:27 2022] PUT /identity/v3/projects/dd6904f93698498c9e72af4a110ce16a/users/6807a8c7d5e04dd9adb155bd2dad4c2b/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:27.838626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1d1a4-5fa2-4e45-935f-fd19aad4d856 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.840796 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7cc1d1a4-5fa2-4e45-935f-fd19aad4d856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:27.846424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1313/2594] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:27 2022] DELETE /identity/v3/users/2125ab5feb514ea1a1dad2e4e940f0bd => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:27.850747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba72b7-6fa1-4413-a25f-0aade8246bb5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.851105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba72b7-6fa1-4413-a25f-0aade8246bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.851454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba72b7-6fa1-4413-a25f-0aade8246bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.870240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-ffba72b7-6fa1-4413-a25f-0aade8246bb5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:27.871509 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ffba72b7-6fa1-4413-a25f-0aade8246bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:27.905794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1d1a4-5fa2-4e45-935f-fd19aad4d856 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.929507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1314/2595] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:27.939659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba72b7-6fa1-4413-a25f-0aade8246bb5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.941012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ffba72b7-6fa1-4413-a25f-0aade8246bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:27.961111 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95786ab-d46f-468a-a4cc-101437a6da42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.961778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95786ab-d46f-468a-a4cc-101437a6da42 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.962058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95786ab-d46f-468a-a4cc-101437a6da42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.962337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95786ab-d46f-468a-a4cc-101437a6da42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.963391 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95786ab-d46f-468a-a4cc-101437a6da42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:27.965621 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95786ab-d46f-468a-a4cc-101437a6da42 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:27.975711 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1282/2596] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:27.979618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95786ab-d46f-468a-a4cc-101437a6da42 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:27.989400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95786ab-d46f-468a-a4cc-101437a6da42 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:27.994659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77a355-d1eb-4ad6-8db5-a5d0e0006eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:27.995568 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77a355-d1eb-4ad6-8db5-a5d0e0006eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:27.995850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77a355-d1eb-4ad6-8db5-a5d0e0006eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:27.996126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77a355-d1eb-4ad6-8db5-a5d0e0006eac admin admin] PATH_INFO: `/v3/projects/dd6904f93698498c9e72af4a110ce16a/users/6807a8c7d5e04dd9adb155bd2dad4c2b/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:27.998828 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77a355-d1eb-4ad6-8db5-a5d0e0006eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6904f93698498c9e72af4a110ce16a, user_id=6807a8c7d5e04dd9adb155bd2dad4c2b, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:28.001262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba72b7-6fa1-4413-a25f-0aade8246bb5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:28.001456 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77a355-d1eb-4ad6-8db5-a5d0e0006eac admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:28.004848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d69fd-43f6-491d-abef-76869b0b07fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:28.005500 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d69fd-43f6-491d-abef-76869b0b07fd service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:28.005791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d69fd-43f6-491d-abef-76869b0b07fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:28.006070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d69fd-43f6-491d-abef-76869b0b07fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:28.012528 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1283/2597] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:27 2022] PUT /identity/v3/projects/dd6904f93698498c9e72af4a110ce16a/users/6807a8c7d5e04dd9adb155bd2dad4c2b/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:28.015753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d69fd-43f6-491d-abef-76869b0b07fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:28.017226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d69fd-43f6-491d-abef-76869b0b07fd service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:28.023976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc26e3f-b2c7-412b-9e34-e889fa05bafe None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:28.024272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc26e3f-b2c7-412b-9e34-e889fa05bafe None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:28.024561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc26e3f-b2c7-412b-9e34-e889fa05bafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:28.027750 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1315/2598] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:49:28.049584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:28.053025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-acc26e3f-b2c7-412b-9e34-e889fa05bafe None None] MFA Rules not processed for user `6807a8c7d5e04dd9adb155bd2dad4c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:28.054674 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-acc26e3f-b2c7-412b-9e34-e889fa05bafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:28.136682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1284/2599] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:28.147272 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d69fd-43f6-491d-abef-76869b0b07fd service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:28.148739 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-141d69fd-43f6-491d-abef-76869b0b07fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:28.216832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d69fd-43f6-491d-abef-76869b0b07fd service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:28.268260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9130c9d-f499-4d73-ae42-da556eed104b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:28.268975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9130c9d-f499-4d73-ae42-da556eed104b service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:28.269257 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9130c9d-f499-4d73-ae42-da556eed104b service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:28.269496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9130c9d-f499-4d73-ae42-da556eed104b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:28.275667 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:28.279635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9130c9d-f499-4d73-ae42-da556eed104b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:28.281335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9130c9d-f499-4d73-ae42-da556eed104b service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:28.291556 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1316/2600] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:28.304608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc26e3f-b2c7-412b-9e34-e889fa05bafe None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:28.306188 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-acc26e3f-b2c7-412b-9e34-e889fa05bafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:28.382993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc26e3f-b2c7-412b-9e34-e889fa05bafe None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:28.387653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:28.388567 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:28.388801 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:28.389072 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] PATH_INFO: `/v3/projects/e9b747fb6a4246bf949404b624f50ec0` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:28.390283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9b747fb6a4246bf949404b624f50ec0)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:28.392223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:28.394318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9130c9d-f499-4d73-ae42-da556eed104b service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:28.402912 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9130c9d-f499-4d73-ae42-da556eed104b service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:28.406524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144c7f1-08f3-4ef1-99aa-fd9934c9ace2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:28.407196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144c7f1-08f3-4ef1-99aa-fd9934c9ace2 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:28.407485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144c7f1-08f3-4ef1-99aa-fd9934c9ace2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:28.407755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144c7f1-08f3-4ef1-99aa-fd9934c9ace2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:28.409779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-5144c7f1-08f3-4ef1-99aa-fd9934c9ace2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:28.441319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9b747fb6a4246bf949404b624f50ec0', 'request_id': 'req-16399036-76c0-4e3c-a5f3-a246f15a2e76'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:28.447701 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9b747fb6a4246bf949404b624f50ec0', 'request_id': 'req-16399036-76c0-4e3c-a5f3-a246f15a2e76'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:28.453886 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9b747fb6a4246bf949404b624f50ec0', 'request_id': 'req-16399036-76c0-4e3c-a5f3-a246f15a2e76'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:28.471048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] The token cache is being invalidate because project e9b747fb6a4246bf949404b624f50ec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:28.471337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:28.472066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:28.473640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1285/2601] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:28 2022] DELETE /identity/v3/projects/e9b747fb6a4246bf949404b624f50ec0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:28.475532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144c7f1-08f3-4ef1-99aa-fd9934c9ace2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:28.477268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144c7f1-08f3-4ef1-99aa-fd9934c9ace2 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:28.477486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:28.479108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:28.486571 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1317/2602] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:28.511489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:28.547919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16399036-76c0-4e3c-a5f3-a246f15a2e76 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:28.553170 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdf2aa-7531-426e-acd0-7df8fa37227d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:28.554215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdf2aa-7531-426e-acd0-7df8fa37227d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:28.554536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdf2aa-7531-426e-acd0-7df8fa37227d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:28.554846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdf2aa-7531-426e-acd0-7df8fa37227d admin admin] PATH_INFO: `/v3/users/cf8f49f0a537414d892c2ef000ae8c5c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:28.556157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdf2aa-7531-426e-acd0-7df8fa37227d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf8f49f0a537414d892c2ef000ae8c5c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:28.558377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdf2aa-7531-426e-acd0-7df8fa37227d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:28.609928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bcfdf2aa-7531-426e-acd0-7df8fa37227d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf8f49f0a537414d892c2ef000ae8c5c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:28.610630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bcfdf2aa-7531-426e-acd0-7df8fa37227d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf8f49f0a537414d892c2ef000ae8c5c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:28.623432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bcfdf2aa-7531-426e-acd0-7df8fa37227d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf8f49f0a537414d892c2ef000ae8c5c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:28.624127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bcfdf2aa-7531-426e-acd0-7df8fa37227d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf8f49f0a537414d892c2ef000ae8c5c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:28.633240 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bcfdf2aa-7531-426e-acd0-7df8fa37227d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf8f49f0a537414d892c2ef000ae8c5c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:28.647512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1286/2603] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:28 2022] DELETE /identity/v3/users/cf8f49f0a537414d892c2ef000ae8c5c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:28.932680 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144c7f1-08f3-4ef1-99aa-fd9934c9ace2 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:28.944404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144c7f1-08f3-4ef1-99aa-fd9934c9ace2 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:29.001723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:29.002594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.002906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.003178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] PATH_INFO: `/v3/projects/9d6fa73910364585adfcc6d46294b255` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.004375 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d6fa73910364585adfcc6d46294b255)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:29.006291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:29.056761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d6fa73910364585adfcc6d46294b255', 'request_id': 'req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:29.062903 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d6fa73910364585adfcc6d46294b255', 'request_id': 'req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:29.068503 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d6fa73910364585adfcc6d46294b255', 'request_id': 'req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:29.083377 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] The token cache is being invalidate because project 9d6fa73910364585adfcc6d46294b255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:29.083664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:29.084356 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a8d3a8fb-3ba2-4cbd-9912-6aacb3916375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:29.085996 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1318/2604] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:28 2022] DELETE /identity/v3/projects/9d6fa73910364585adfcc6d46294b255 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:29.092611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276e53a-bd7c-43f3-bc68-4a880d68f4ae None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.093108 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276e53a-bd7c-43f3-bc68-4a880d68f4ae None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.093206 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276e53a-bd7c-43f3-bc68-4a880d68f4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.116402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c276e53a-bd7c-43f3-bc68-4a880d68f4ae None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:29.117604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c276e53a-bd7c-43f3-bc68-4a880d68f4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:29.176949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1287/2605] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:29.190793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d529df7-e012-462a-bb13-2434104611d8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.191179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d529df7-e012-462a-bb13-2434104611d8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.191746 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d529df7-e012-462a-bb13-2434104611d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.212404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-1d529df7-e012-462a-bb13-2434104611d8 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:29.213627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-1d529df7-e012-462a-bb13-2434104611d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:29.222969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1319/2606] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:29.227335 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276e53a-bd7c-43f3-bc68-4a880d68f4ae None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:29.239393 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276e53a-bd7c-43f3-bc68-4a880d68f4ae None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:29.248652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2c87f-1999-4911-bde0-1b6cfcd7d08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:29.249624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c87f-1999-4911-bde0-1b6cfcd7d08a admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.249996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c87f-1999-4911-bde0-1b6cfcd7d08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.250342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2c87f-1999-4911-bde0-1b6cfcd7d08a admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.251520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2c87f-1999-4911-bde0-1b6cfcd7d08a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:29.251927 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2c87f-1999-4911-bde0-1b6cfcd7d08a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:29.254359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2c87f-1999-4911-bde0-1b6cfcd7d08a admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:29.268588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1288/2607] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:49:29.272994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d529df7-e012-462a-bb13-2434104611d8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:29.282744 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d529df7-e012-462a-bb13-2434104611d8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:29.288508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d1726-b61b-4601-8f0a-eb0e1600e557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:29.289168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d1726-b61b-4601-8f0a-eb0e1600e557 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.289453 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d1726-b61b-4601-8f0a-eb0e1600e557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.289726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d1726-b61b-4601-8f0a-eb0e1600e557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.290420 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d1726-b61b-4601-8f0a-eb0e1600e557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:29.292589 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894d1726-b61b-4601-8f0a-eb0e1600e557 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:29.306547 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:29.322343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1320/2608] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:29 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:29.326489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2c87f-1999-4911-bde0-1b6cfcd7d08a admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:29.336957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2c87f-1999-4911-bde0-1b6cfcd7d08a admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:29.372249 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:29.399536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4198ea0-0aaa-48e7-afea-56d2ae8d344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:29.400386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4198ea0-0aaa-48e7-afea-56d2ae8d344b admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.400684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4198ea0-0aaa-48e7-afea-56d2ae8d344b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.400969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4198ea0-0aaa-48e7-afea-56d2ae8d344b admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.401669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4198ea0-0aaa-48e7-afea-56d2ae8d344b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:29.403661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4198ea0-0aaa-48e7-afea-56d2ae8d344b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:29.425572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1289/2609] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:29 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:29.430051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d1726-b61b-4601-8f0a-eb0e1600e557 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:29.441190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d1726-b61b-4601-8f0a-eb0e1600e557 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:29.446115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba75e2-7ac5-4ada-8724-555ead04851c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:29.446827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba75e2-7ac5-4ada-8724-555ead04851c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.447118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba75e2-7ac5-4ada-8724-555ead04851c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.447493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba75e2-7ac5-4ada-8724-555ead04851c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.448167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba75e2-7ac5-4ada-8724-555ead04851c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:29.449782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba75e2-7ac5-4ada-8724-555ead04851c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:29.460653 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1321/2610] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:29.465446 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4198ea0-0aaa-48e7-afea-56d2ae8d344b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:29.476153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4198ea0-0aaa-48e7-afea-56d2ae8d344b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:29.481735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dc713-faf5-4298-92ed-f5f3f5d5b979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:29.482405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dc713-faf5-4298-92ed-f5f3f5d5b979 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.482728 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dc713-faf5-4298-92ed-f5f3f5d5b979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.483002 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dc713-faf5-4298-92ed-f5f3f5d5b979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.483748 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9dc713-faf5-4298-92ed-f5f3f5d5b979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:29.485390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9dc713-faf5-4298-92ed-f5f3f5d5b979 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:29.495972 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1290/2611] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:29.500406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba75e2-7ac5-4ada-8724-555ead04851c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:29.511636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba75e2-7ac5-4ada-8724-555ead04851c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:29.517805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627e5c6-469f-4f1a-a859-c0449da559fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:29.518845 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627e5c6-469f-4f1a-a859-c0449da559fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.519132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627e5c6-469f-4f1a-a859-c0449da559fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.519405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627e5c6-469f-4f1a-a859-c0449da559fb admin admin] PATH_INFO: `/v3/projects/1e6175ae5d9b427683f3090116198776/users/4a8ff6146af3441ba7e67a00c5f39857/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.535726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627e5c6-469f-4f1a-a859-c0449da559fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6175ae5d9b427683f3090116198776, user_id=4a8ff6146af3441ba7e67a00c5f39857, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:29.538424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627e5c6-469f-4f1a-a859-c0449da559fb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:29.554454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1322/2612] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:29 2022] PUT /identity/v3/projects/1e6175ae5d9b427683f3090116198776/users/4a8ff6146af3441ba7e67a00c5f39857/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:29.565362 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53d7e1-a81b-47ab-9dac-aaf49606833e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.565654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53d7e1-a81b-47ab-9dac-aaf49606833e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.565942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53d7e1-a81b-47ab-9dac-aaf49606833e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.594891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-7e53d7e1-a81b-47ab-9dac-aaf49606833e None None] MFA Rules not processed for user `4a8ff6146af3441ba7e67a00c5f39857`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:29.596127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7e53d7e1-a81b-47ab-9dac-aaf49606833e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:29.662963 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1291/2613] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:29.667974 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627e5c6-469f-4f1a-a859-c0449da559fb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:29.730515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627e5c6-469f-4f1a-a859-c0449da559fb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:29.735689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9207ca-b097-4da0-8ba1-65576a7b75f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:29.736347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9207ca-b097-4da0-8ba1-65576a7b75f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.736639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9207ca-b097-4da0-8ba1-65576a7b75f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.736906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9207ca-b097-4da0-8ba1-65576a7b75f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.737593 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9207ca-b097-4da0-8ba1-65576a7b75f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:29.739550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9207ca-b097-4da0-8ba1-65576a7b75f7 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:29.766512 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1323/2614] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:29 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:29.770652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53d7e1-a81b-47ab-9dac-aaf49606833e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:29.782081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53d7e1-a81b-47ab-9dac-aaf49606833e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:29.847904 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6ffca-8311-47e1-804e-0155c3bf07fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:29.848598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ffca-8311-47e1-804e-0155c3bf07fa admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.848882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ffca-8311-47e1-804e-0155c3bf07fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.849157 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ffca-8311-47e1-804e-0155c3bf07fa admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.849930 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6ffca-8311-47e1-804e-0155c3bf07fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:29.851887 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6ffca-8311-47e1-804e-0155c3bf07fa admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:29.881432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1292/2615] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:29 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:29.881698 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9207ca-b097-4da0-8ba1-65576a7b75f7 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:29.889311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9207ca-b097-4da0-8ba1-65576a7b75f7 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:29.895136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e8f37-ec77-4b51-9fa0-36bec4428e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:29.895861 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e8f37-ec77-4b51-9fa0-36bec4428e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.896140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e8f37-ec77-4b51-9fa0-36bec4428e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.896409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e8f37-ec77-4b51-9fa0-36bec4428e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.897139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e8f37-ec77-4b51-9fa0-36bec4428e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:29.898752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e8f37-ec77-4b51-9fa0-36bec4428e6c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:29.916006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1324/2616] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:29.917378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6ffca-8311-47e1-804e-0155c3bf07fa admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:29.928078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6ffca-8311-47e1-804e-0155c3bf07fa admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:29.933668 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4a405-50f6-4a0b-90db-dc089310df26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:29.934606 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4a405-50f6-4a0b-90db-dc089310df26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:29.934928 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4a405-50f6-4a0b-90db-dc089310df26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:29.935207 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4a405-50f6-4a0b-90db-dc089310df26 admin admin] PATH_INFO: `/v3/projects/03d83a1845894af99d4a84bc8c899d1f/users/0389ced4be584247a2df4072f04e9516/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:29.951839 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4a405-50f6-4a0b-90db-dc089310df26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d83a1845894af99d4a84bc8c899d1f, user_id=0389ced4be584247a2df4072f04e9516, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:29.954542 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4a405-50f6-4a0b-90db-dc089310df26 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:29.970934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1293/2617] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:29 2022] PUT /identity/v3/projects/03d83a1845894af99d4a84bc8c899d1f/users/0389ced4be584247a2df4072f04e9516/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:29.975253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e8f37-ec77-4b51-9fa0-36bec4428e6c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:29.986137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e8f37-ec77-4b51-9fa0-36bec4428e6c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:30.057854 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636b1d8-1e21-4799-a662-813b256f47fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:30.058531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636b1d8-1e21-4799-a662-813b256f47fd admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:30.058843 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636b1d8-1e21-4799-a662-813b256f47fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:30.059118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636b1d8-1e21-4799-a662-813b256f47fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:30.059850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636b1d8-1e21-4799-a662-813b256f47fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:30.061443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636b1d8-1e21-4799-a662-813b256f47fd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:30.071900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1325/2618] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:30.076107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4a405-50f6-4a0b-90db-dc089310df26 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:30.086962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4a405-50f6-4a0b-90db-dc089310df26 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:30.092406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f783261-6b55-40cd-9d5c-f6d5d5f6e577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:30.093403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783261-6b55-40cd-9d5c-f6d5d5f6e577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:30.093990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783261-6b55-40cd-9d5c-f6d5d5f6e577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:30.093990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783261-6b55-40cd-9d5c-f6d5d5f6e577 admin admin] PATH_INFO: `/v3/projects/03d83a1845894af99d4a84bc8c899d1f/users/0389ced4be584247a2df4072f04e9516/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:30.096191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f783261-6b55-40cd-9d5c-f6d5d5f6e577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d83a1845894af99d4a84bc8c899d1f, user_id=0389ced4be584247a2df4072f04e9516, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:30.099097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f783261-6b55-40cd-9d5c-f6d5d5f6e577 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:30.102629 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:30.112024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1294/2619] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:30 2022] PUT /identity/v3/projects/03d83a1845894af99d4a84bc8c899d1f/users/0389ced4be584247a2df4072f04e9516/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:30.125063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035abe3d-1a7e-4ea7-b8be-627faae95c74 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:30.125354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035abe3d-1a7e-4ea7-b8be-627faae95c74 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:30.125628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035abe3d-1a7e-4ea7-b8be-627faae95c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:30.152392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-035abe3d-1a7e-4ea7-b8be-627faae95c74 None None] MFA Rules not processed for user `0389ced4be584247a2df4072f04e9516`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:30.153606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-035abe3d-1a7e-4ea7-b8be-627faae95c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:30.171554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:30.227525 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1326/2620] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:30.239321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f783261-6b55-40cd-9d5c-f6d5d5f6e577 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:30.241132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8f783261-6b55-40cd-9d5c-f6d5d5f6e577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:30.302985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f783261-6b55-40cd-9d5c-f6d5d5f6e577 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:30.307080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8feb6-6e19-40a0-ab89-813c3788a0ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:30.307736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8feb6-6e19-40a0-ab89-813c3788a0ad service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:30.308016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8feb6-6e19-40a0-ab89-813c3788a0ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:30.308288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8feb6-6e19-40a0-ab89-813c3788a0ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:30.318502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8feb6-6e19-40a0-ab89-813c3788a0ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:30.320450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8feb6-6e19-40a0-ab89-813c3788a0ad service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:30.330950 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1295/2621] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:30.359936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035abe3d-1a7e-4ea7-b8be-627faae95c74 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:30.422623 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035abe3d-1a7e-4ea7-b8be-627faae95c74 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:30.427388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:30.428274 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:30.428553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:30.428827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] PATH_INFO: `/v3/users/4a8ff6146af3441ba7e67a00c5f39857` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:30.430166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a8ff6146af3441ba7e67a00c5f39857)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:30.432262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:30.488155 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a8ff6146af3441ba7e67a00c5f39857'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:30.488817 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a8ff6146af3441ba7e67a00c5f39857'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:30.503673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a8ff6146af3441ba7e67a00c5f39857'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:30.504438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a8ff6146af3441ba7e67a00c5f39857'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:30.515947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a8ff6146af3441ba7e67a00c5f39857'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:30.528502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1327/2622] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:30 2022] DELETE /identity/v3/users/4a8ff6146af3441ba7e67a00c5f39857 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:30.533104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f06d33-43bc-433e-ac8f-dfa560baec55 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:30.533394 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f06d33-43bc-433e-ac8f-dfa560baec55 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:30.533672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f06d33-43bc-433e-ac8f-dfa560baec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:30.551405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f4f06d33-43bc-433e-ac8f-dfa560baec55 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:30.552624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f4f06d33-43bc-433e-ac8f-dfa560baec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:30.617792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1296/2623] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:30.628097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:30.629742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:30.675082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:30.698856 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc677a-347b-4d14-9b41-d7c4e88ff282 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:30.702651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938676b-0f32-4529-98e0-1fd253a8710d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:30.703337 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938676b-0f32-4529-98e0-1fd253a8710d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:30.703616 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938676b-0f32-4529-98e0-1fd253a8710d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:30.703892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938676b-0f32-4529-98e0-1fd253a8710d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:30.714448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938676b-0f32-4529-98e0-1fd253a8710d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:30.715929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938676b-0f32-4529-98e0-1fd253a8710d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:30.725590 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1328/2624] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:31.015835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f06d33-43bc-433e-ac8f-dfa560baec55 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:31.017463 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f4f06d33-43bc-433e-ac8f-dfa560baec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:31.031634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f06d33-43bc-433e-ac8f-dfa560baec55 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:31.038613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:31.039554 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:31.039815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:31.040087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] PATH_INFO: `/v3/projects/1e6175ae5d9b427683f3090116198776` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:31.041293 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e6175ae5d9b427683f3090116198776)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:31.043460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:31.072786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938676b-0f32-4529-98e0-1fd253a8710d service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:31.074298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-3938676b-0f32-4529-98e0-1fd253a8710d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:31.089266 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938676b-0f32-4529-98e0-1fd253a8710d service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:31.100743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e6175ae5d9b427683f3090116198776', 'request_id': 'req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.107358 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e6175ae5d9b427683f3090116198776', 'request_id': 'req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.113003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e6175ae5d9b427683f3090116198776', 'request_id': 'req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.125643 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] The token cache is being invalidate because project 1e6175ae5d9b427683f3090116198776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:31.125915 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.126535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.128259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1297/2625] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:31 2022] DELETE /identity/v3/projects/1e6175ae5d9b427683f3090116198776 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:31.132121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:31.133556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:31.151341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae40102-f0a7-4e8e-8529-c6452742207a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:31.152399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae40102-f0a7-4e8e-8529-c6452742207a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:31.152797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae40102-f0a7-4e8e-8529-c6452742207a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:31.153078 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae40102-f0a7-4e8e-8529-c6452742207a admin admin] PATH_INFO: `/v3/users/9771c1b7166541bcbea0c3dee75cd339` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:31.154317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae40102-f0a7-4e8e-8529-c6452742207a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9771c1b7166541bcbea0c3dee75cd339)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:31.156312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae40102-f0a7-4e8e-8529-c6452742207a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:31.161277 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:31.204202 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f95197-f7cd-4c7b-a11e-1b0b2eef4250 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:31.205645 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fae40102-f0a7-4e8e-8529-c6452742207a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9771c1b7166541bcbea0c3dee75cd339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.206279 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fae40102-f0a7-4e8e-8529-c6452742207a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9771c1b7166541bcbea0c3dee75cd339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.208798 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:31.209682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:31.209962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:31.210234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] PATH_INFO: `/v3/users/0389ced4be584247a2df4072f04e9516` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:31.211473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0389ced4be584247a2df4072f04e9516)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:31.213489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:31.219228 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fae40102-f0a7-4e8e-8529-c6452742207a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9771c1b7166541bcbea0c3dee75cd339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.219975 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fae40102-f0a7-4e8e-8529-c6452742207a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9771c1b7166541bcbea0c3dee75cd339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.229424 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-fae40102-f0a7-4e8e-8529-c6452742207a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9771c1b7166541bcbea0c3dee75cd339'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.243539 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1329/2626] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:31 2022] DELETE /identity/v3/users/9771c1b7166541bcbea0c3dee75cd339 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:31.248632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a477359-061b-4e29-9c5d-cc9a57032363 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:31.249010 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a477359-061b-4e29-9c5d-cc9a57032363 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:31.249365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a477359-061b-4e29-9c5d-cc9a57032363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:31.268318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-2a477359-061b-4e29-9c5d-cc9a57032363 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:31.269533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2a477359-061b-4e29-9c5d-cc9a57032363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:31.272028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0389ced4be584247a2df4072f04e9516'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.272717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0389ced4be584247a2df4072f04e9516'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.285357 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0389ced4be584247a2df4072f04e9516'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.286232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0389ced4be584247a2df4072f04e9516'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.294518 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0389ced4be584247a2df4072f04e9516'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.307531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1298/2627] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:31 2022] DELETE /identity/v3/users/0389ced4be584247a2df4072f04e9516 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:31.320666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:31.334350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1330/2628] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:31.346525 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:31.348027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:31.409672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9975967-975c-4ce9-8664-a250c3659684 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:31.414962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f72b45-6d98-4a65-ab3f-a446814bc7d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:31.415628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f72b45-6d98-4a65-ab3f-a446814bc7d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:31.416174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f72b45-6d98-4a65-ab3f-a446814bc7d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:31.416477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f72b45-6d98-4a65-ab3f-a446814bc7d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:31.492908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:31.505343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f72b45-6d98-4a65-ab3f-a446814bc7d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:31.507422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f72b45-6d98-4a65-ab3f-a446814bc7d7 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:31.519082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1299/2629] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:31.597371 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a477359-061b-4e29-9c5d-cc9a57032363 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:31.598936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2a477359-061b-4e29-9c5d-cc9a57032363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:31.621959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a477359-061b-4e29-9c5d-cc9a57032363 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:31.627640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:31.628609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:31.628914 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:31.629191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] PATH_INFO: `/v3/projects/03d83a1845894af99d4a84bc8c899d1f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:31.630402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] RBAC: Authorizing `identity:delete_project(project_id=03d83a1845894af99d4a84bc8c899d1f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:31.632403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:31.684281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03d83a1845894af99d4a84bc8c899d1f', 'request_id': 'req-f46ad6d0-2c60-4732-ae2a-d9808683733b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.691511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03d83a1845894af99d4a84bc8c899d1f', 'request_id': 'req-f46ad6d0-2c60-4732-ae2a-d9808683733b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.698608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03d83a1845894af99d4a84bc8c899d1f', 'request_id': 'req-f46ad6d0-2c60-4732-ae2a-d9808683733b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.713005 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] The token cache is being invalidate because project 03d83a1845894af99d4a84bc8c899d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:31.713301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.714862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.716517 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1331/2630] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:31 2022] DELETE /identity/v3/projects/03d83a1845894af99d4a84bc8c899d1f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:31.721457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc064f02-8c0c-493a-b149-38b453c2ac1f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:31.721752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc064f02-8c0c-493a-b149-38b453c2ac1f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:31.722035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc064f02-8c0c-493a-b149-38b453c2ac1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:31.740305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-fc064f02-8c0c-493a-b149-38b453c2ac1f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:31.741519 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-fc064f02-8c0c-493a-b149-38b453c2ac1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:31.805482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1300/2631] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:31.809704 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:31.811221 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:31.813196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf54afc-c7de-4a13-b02c-0ab06a254407 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:31.813733 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf54afc-c7de-4a13-b02c-0ab06a254407 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:31.814116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf54afc-c7de-4a13-b02c-0ab06a254407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:31.825472 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ad6d0-2c60-4732-ae2a-d9808683733b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:31.836626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:31.837952 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:31.838309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:31.838504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] PATH_INFO: `/v3/projects/0742f9481a2d41228f2f72fd62ecbf82` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:31.840640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0742f9481a2d41228f2f72fd62ecbf82)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:31.842773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:31.843156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-edf54afc-c7de-4a13-b02c-0ab06a254407 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:31.844395 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-edf54afc-c7de-4a13-b02c-0ab06a254407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:31.852630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1301/2632] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:31.857050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf54afc-c7de-4a13-b02c-0ab06a254407 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:31.866962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf54afc-c7de-4a13-b02c-0ab06a254407 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:31.872482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f2da2-d06f-452c-b6f4-9f309dd93328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:31.873220 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2da2-d06f-452c-b6f4-9f309dd93328 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:31.873578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2da2-d06f-452c-b6f4-9f309dd93328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:31.873793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f2da2-d06f-452c-b6f4-9f309dd93328 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:31.874719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f2da2-d06f-452c-b6f4-9f309dd93328 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:31.875071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f2da2-d06f-452c-b6f4-9f309dd93328 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:31.876761 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f2da2-d06f-452c-b6f4-9f309dd93328 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:31.893228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1302/2633] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:49:31.894716 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f2da2-d06f-452c-b6f4-9f309dd93328 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:31.898720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0742f9481a2d41228f2f72fd62ecbf82', 'request_id': 'req-b7585f19-25d2-4254-a20a-d6604aa86d92'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.905259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0742f9481a2d41228f2f72fd62ecbf82', 'request_id': 'req-b7585f19-25d2-4254-a20a-d6604aa86d92'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.905734 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f2da2-d06f-452c-b6f4-9f309dd93328 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:31.911643 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0742f9481a2d41228f2f72fd62ecbf82', 'request_id': 'req-b7585f19-25d2-4254-a20a-d6604aa86d92'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.925729 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] The token cache is being invalidate because project 0742f9481a2d41228f2f72fd62ecbf82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:31.926012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.926646 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:31.928467 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1332/2634] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:31 2022] DELETE /identity/v3/projects/0742f9481a2d41228f2f72fd62ecbf82 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:31.932373 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:31.933635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:31.965125 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837ad55-59a0-4b30-acfb-b9f53a4a0fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:31.965960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837ad55-59a0-4b30-acfb-b9f53a4a0fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:31.966341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837ad55-59a0-4b30-acfb-b9f53a4a0fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:31.966620 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837ad55-59a0-4b30-acfb-b9f53a4a0fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:31.967349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837ad55-59a0-4b30-acfb-b9f53a4a0fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:31.969295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2837ad55-59a0-4b30-acfb-b9f53a4a0fa8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:31.999207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7585f19-25d2-4254-a20a-d6604aa86d92 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.001461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1303/2635] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:31 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:32.003926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:32.005961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837ad55-59a0-4b30-acfb-b9f53a4a0fa8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.007776 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2837ad55-59a0-4b30-acfb-b9f53a4a0fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:32.059999 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.060889 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.061169 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.061443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] PATH_INFO: `/v3/users/6807a8c7d5e04dd9adb155bd2dad4c2b` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.062659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6807a8c7d5e04dd9adb155bd2dad4c2b)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.064737 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.067852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:32.071640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837ad55-59a0-4b30-acfb-b9f53a4a0fa8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.077630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79552eb7-6318-4afd-986b-4bf333e29f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.078345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79552eb7-6318-4afd-986b-4bf333e29f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.078893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79552eb7-6318-4afd-986b-4bf333e29f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.079031 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79552eb7-6318-4afd-986b-4bf333e29f7f admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.079630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79552eb7-6318-4afd-986b-4bf333e29f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.081559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79552eb7-6318-4afd-986b-4bf333e29f7f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.107214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1304/2636] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:32 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:32.111641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79552eb7-6318-4afd-986b-4bf333e29f7f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.121399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6807a8c7d5e04dd9adb155bd2dad4c2b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:32.122104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79552eb7-6318-4afd-986b-4bf333e29f7f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.122335 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6807a8c7d5e04dd9adb155bd2dad4c2b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:32.126758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7f2af-8ea7-41ba-9303-382531738a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.127426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7f2af-8ea7-41ba-9303-382531738a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.127715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7f2af-8ea7-41ba-9303-382531738a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.127998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7f2af-8ea7-41ba-9303-382531738a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.128730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7f2af-8ea7-41ba-9303-382531738a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.130360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be7f2af-8ea7-41ba-9303-382531738a2c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.135084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6807a8c7d5e04dd9adb155bd2dad4c2b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:32.135794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6807a8c7d5e04dd9adb155bd2dad4c2b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:32.141640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1305/2637] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:32.144442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6807a8c7d5e04dd9adb155bd2dad4c2b'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:32.145873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7f2af-8ea7-41ba-9303-382531738a2c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.147838 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9be7f2af-8ea7-41ba-9303-382531738a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:32.157586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1333/2638] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:31 2022] DELETE /identity/v3/users/6807a8c7d5e04dd9adb155bd2dad4c2b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:32.161045 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7f2af-8ea7-41ba-9303-382531738a2c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.220652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba9567-a4cc-4f4c-b89a-e1077ae4e515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.221335 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba9567-a4cc-4f4c-b89a-e1077ae4e515 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.221618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba9567-a4cc-4f4c-b89a-e1077ae4e515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.221893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba9567-a4cc-4f4c-b89a-e1077ae4e515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.222619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba9567-a4cc-4f4c-b89a-e1077ae4e515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.224250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba9567-a4cc-4f4c-b89a-e1077ae4e515 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.239017 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1306/2639] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:32.240833 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.251817 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afec7c-7807-4b9c-ab10-77eb82234155 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.256474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c96c7-8c90-4601-baca-edcbe9e7b1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.257372 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c96c7-8c90-4601-baca-edcbe9e7b1ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.257659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c96c7-8c90-4601-baca-edcbe9e7b1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.257944 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c96c7-8c90-4601-baca-edcbe9e7b1ff admin admin] PATH_INFO: `/v3/projects/7667209b909a4f56a89d1be06ced4431/users/2f6be4677f1c441c8df4d079a1f38679/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.274393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c96c7-8c90-4601-baca-edcbe9e7b1ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=7667209b909a4f56a89d1be06ced4431, user_id=2f6be4677f1c441c8df4d079a1f38679, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.277759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c96c7-8c90-4601-baca-edcbe9e7b1ff admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.293080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1334/2640] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:32 2022] PUT /identity/v3/projects/7667209b909a4f56a89d1be06ced4431/users/2f6be4677f1c441c8df4d079a1f38679/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:32.303155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bce5f-9f7c-422e-856d-dd5d0ca8b71e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.303451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bce5f-9f7c-422e-856d-dd5d0ca8b71e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.303731 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bce5f-9f7c-422e-856d-dd5d0ca8b71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.333938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-366bce5f-9f7c-422e-856d-dd5d0ca8b71e None None] MFA Rules not processed for user `2f6be4677f1c441c8df4d079a1f38679`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:32.335197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-366bce5f-9f7c-422e-856d-dd5d0ca8b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:32.399342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1307/2641] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:32.404019 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c96c7-8c90-4601-baca-edcbe9e7b1ff admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.450048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bce5f-9f7c-422e-856d-dd5d0ca8b71e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.452482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-366bce5f-9f7c-422e-856d-dd5d0ca8b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:32.490563 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c96c7-8c90-4601-baca-edcbe9e7b1ff admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.496310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623e8fc-11f9-428a-a58a-571fc3a5f422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.496980 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623e8fc-11f9-428a-a58a-571fc3a5f422 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.497263 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623e8fc-11f9-428a-a58a-571fc3a5f422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.497540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623e8fc-11f9-428a-a58a-571fc3a5f422 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.498265 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623e8fc-11f9-428a-a58a-571fc3a5f422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.500204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9623e8fc-11f9-428a-a58a-571fc3a5f422 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.509027 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:32.530017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1335/2642] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:32 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:32.534267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623e8fc-11f9-428a-a58a-571fc3a5f422 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.534425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bce5f-9f7c-422e-856d-dd5d0ca8b71e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.539669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:32.595959 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.596869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.597185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.597466 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] PATH_INFO: `/v3/projects/dd6904f93698498c9e72af4a110ce16a` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.598832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd6904f93698498c9e72af4a110ce16a)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.599901 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623e8fc-11f9-428a-a58a-571fc3a5f422 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.600787 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.604869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7b96f-e32c-4c57-9ea6-6fb1331caf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.605541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7b96f-e32c-4c57-9ea6-6fb1331caf7e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.605825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7b96f-e32c-4c57-9ea6-6fb1331caf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.606098 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7b96f-e32c-4c57-9ea6-6fb1331caf7e admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.606819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7b96f-e32c-4c57-9ea6-6fb1331caf7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.608729 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7b96f-e32c-4c57-9ea6-6fb1331caf7e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.632046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1336/2643] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:32 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:32.636604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7b96f-e32c-4c57-9ea6-6fb1331caf7e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.646927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7b96f-e32c-4c57-9ea6-6fb1331caf7e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.650854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd6904f93698498c9e72af4a110ce16a', 'request_id': 'req-d6d68f6f-7314-4cef-9c02-79f6a335db03'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:32.657133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd6904f93698498c9e72af4a110ce16a', 'request_id': 'req-d6d68f6f-7314-4cef-9c02-79f6a335db03'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:32.662806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd6904f93698498c9e72af4a110ce16a', 'request_id': 'req-d6d68f6f-7314-4cef-9c02-79f6a335db03'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:32.676491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] The token cache is being invalidate because project dd6904f93698498c9e72af4a110ce16a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:32.676775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:32.677415 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d6d68f6f-7314-4cef-9c02-79f6a335db03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:32.679074 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1308/2644] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:32 2022] DELETE /identity/v3/projects/dd6904f93698498c9e72af4a110ce16a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:32.684059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e910e49-83c3-45b4-a7cd-1bedca4e0169 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.684556 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e910e49-83c3-45b4-a7cd-1bedca4e0169 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.684969 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e910e49-83c3-45b4-a7cd-1bedca4e0169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.703317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8e910e49-83c3-45b4-a7cd-1bedca4e0169 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:32.704562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8e910e49-83c3-45b4-a7cd-1bedca4e0169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:32.707394 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012282f-851c-4c8f-9922-3e5307103149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.708076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012282f-851c-4c8f-9922-3e5307103149 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.708361 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012282f-851c-4c8f-9922-3e5307103149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.708640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012282f-851c-4c8f-9922-3e5307103149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.709368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012282f-851c-4c8f-9922-3e5307103149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.711013 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012282f-851c-4c8f-9922-3e5307103149 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.713433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1309/2645] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:32.721179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7df0bc-66c3-4e0a-a5a2-e01edf1cac1b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.721469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7df0bc-66c3-4e0a-a5a2-e01edf1cac1b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.721749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7df0bc-66c3-4e0a-a5a2-e01edf1cac1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.729421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1337/2646] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:32.730452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012282f-851c-4c8f-9922-3e5307103149 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.732260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a012282f-851c-4c8f-9922-3e5307103149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:32.741064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ac7df0bc-66c3-4e0a-a5a2-e01edf1cac1b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:32.742331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ac7df0bc-66c3-4e0a-a5a2-e01edf1cac1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:32.744777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012282f-851c-4c8f-9922-3e5307103149 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.749379 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c5fc7-3fbb-428f-b4a4-c59d08c36c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.750037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c5fc7-3fbb-428f-b4a4-c59d08c36c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.750317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c5fc7-3fbb-428f-b4a4-c59d08c36c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.750591 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c5fc7-3fbb-428f-b4a4-c59d08c36c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.751197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1310/2647] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:32.751487 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c5fc7-3fbb-428f-b4a4-c59d08c36c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.753084 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c5fc7-3fbb-428f-b4a4-c59d08c36c7b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.755505 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7df0bc-66c3-4e0a-a5a2-e01edf1cac1b None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.767316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1338/2648] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:32.767480 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7df0bc-66c3-4e0a-a5a2-e01edf1cac1b None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.768318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c5fc7-3fbb-428f-b4a4-c59d08c36c7b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.770080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcfcbd-0677-4a56-9eda-dd5e0f19cc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.770738 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcfcbd-0677-4a56-9eda-dd5e0f19cc18 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.771055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcfcbd-0677-4a56-9eda-dd5e0f19cc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.771305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcfcbd-0677-4a56-9eda-dd5e0f19cc18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.772075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcfcbd-0677-4a56-9eda-dd5e0f19cc18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:32.772382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcfcbd-0677-4a56-9eda-dd5e0f19cc18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.773983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcfcbd-0677-4a56-9eda-dd5e0f19cc18 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.779249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c5fc7-3fbb-428f-b4a4-c59d08c36c7b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.784300 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269a3af-2477-43be-a420-843c398ee53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.785595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269a3af-2477-43be-a420-843c398ee53b admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.785883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269a3af-2477-43be-a420-843c398ee53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.789440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269a3af-2477-43be-a420-843c398ee53b admin admin] PATH_INFO: `/v3/projects/a7567ebed2d2448a8d5dfdbfd9486f7c/users/eefe37811d7d424baac016d6ba959316/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.789574 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1311/2649] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:49:32.790334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcfcbd-0677-4a56-9eda-dd5e0f19cc18 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.800253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcfcbd-0677-4a56-9eda-dd5e0f19cc18 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.802913 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269a3af-2477-43be-a420-843c398ee53b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7567ebed2d2448a8d5dfdbfd9486f7c, user_id=eefe37811d7d424baac016d6ba959316, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.804882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eccb5-2bbd-4412-afdc-41f53fddf05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.805557 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eccb5-2bbd-4412-afdc-41f53fddf05b admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.805700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2269a3af-2477-43be-a420-843c398ee53b admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.805800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eccb5-2bbd-4412-afdc-41f53fddf05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.806091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eccb5-2bbd-4412-afdc-41f53fddf05b admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.806926 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55eccb5-2bbd-4412-afdc-41f53fddf05b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.809170 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55eccb5-2bbd-4412-afdc-41f53fddf05b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.830809 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1339/2650] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:32 2022] PUT /identity/v3/projects/a7567ebed2d2448a8d5dfdbfd9486f7c/users/eefe37811d7d424baac016d6ba959316/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:32.844935 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1312/2651] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:32 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:32.846161 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269a3af-2477-43be-a420-843c398ee53b admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:32.893477 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b307b-9600-4e8c-b0a5-a761bf4d1c39 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.893684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b307b-9600-4e8c-b0a5-a761bf4d1c39 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.893933 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b307b-9600-4e8c-b0a5-a761bf4d1c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.896136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:32.918604 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269a3af-2477-43be-a420-843c398ee53b admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:32.922338 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-2d7b307b-9600-4e8c-b0a5-a761bf4d1c39 None None] MFA Rules not processed for user `eefe37811d7d424baac016d6ba959316`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:32.923579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2d7b307b-9600-4e8c-b0a5-a761bf4d1c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:32.966874 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:32.976308 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81427403-266a-4182-9e09-7ecefd97d66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:32.976981 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81427403-266a-4182-9e09-7ecefd97d66a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:32.977323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81427403-266a-4182-9e09-7ecefd97d66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:32.977595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81427403-266a-4182-9e09-7ecefd97d66a admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:32.978363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81427403-266a-4182-9e09-7ecefd97d66a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:32.980400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81427403-266a-4182-9e09-7ecefd97d66a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:32.995818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1313/2652] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:33.000540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b307b-9600-4e8c-b0a5-a761bf4d1c39 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.002285 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1340/2653] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:32 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:33.006035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81427403-266a-4182-9e09-7ecefd97d66a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.012218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b307b-9600-4e8c-b0a5-a761bf4d1c39 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.016727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81427403-266a-4182-9e09-7ecefd97d66a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.018871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abdc4d-a486-4394-b23f-017e768bba57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.019575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abdc4d-a486-4394-b23f-017e768bba57 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.019849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abdc4d-a486-4394-b23f-017e768bba57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.020116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abdc4d-a486-4394-b23f-017e768bba57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.020810 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abdc4d-a486-4394-b23f-017e768bba57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.021713 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07e034-f8fe-434d-a0c2-d6f0fc9f5602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.022370 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07e034-f8fe-434d-a0c2-d6f0fc9f5602 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.022655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07e034-f8fe-434d-a0c2-d6f0fc9f5602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.022786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abdc4d-a486-4394-b23f-017e768bba57 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.022966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07e034-f8fe-434d-a0c2-d6f0fc9f5602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.023695 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07e034-f8fe-434d-a0c2-d6f0fc9f5602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.025317 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07e034-f8fe-434d-a0c2-d6f0fc9f5602 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.038420 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1341/2654] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:33.039714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07e034-f8fe-434d-a0c2-d6f0fc9f5602 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.050370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1314/2655] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:32 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:33.050779 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07e034-f8fe-434d-a0c2-d6f0fc9f5602 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.054509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abdc4d-a486-4394-b23f-017e768bba57 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.113567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c53414-a9de-497d-9936-9c092d4d0335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.114240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c53414-a9de-497d-9936-9c092d4d0335 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.114527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c53414-a9de-497d-9936-9c092d4d0335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.114840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c53414-a9de-497d-9936-9c092d4d0335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.115602 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c53414-a9de-497d-9936-9c092d4d0335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.117206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c53414-a9de-497d-9936-9c092d4d0335 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.124082 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abdc4d-a486-4394-b23f-017e768bba57 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.127721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1342/2656] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:33.128998 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a73fac-12e3-479b-aebe-3be22b59af76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.129678 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a73fac-12e3-479b-aebe-3be22b59af76 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.129991 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a73fac-12e3-479b-aebe-3be22b59af76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.130273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a73fac-12e3-479b-aebe-3be22b59af76 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.131182 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a73fac-12e3-479b-aebe-3be22b59af76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.131545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c53414-a9de-497d-9936-9c092d4d0335 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.133102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a73fac-12e3-479b-aebe-3be22b59af76 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.141620 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c53414-a9de-497d-9936-9c092d4d0335 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.146513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f96cb-39fa-44b4-b60f-9f040aecb380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.147380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f96cb-39fa-44b4-b60f-9f040aecb380 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.147766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f96cb-39fa-44b4-b60f-9f040aecb380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.148048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f96cb-39fa-44b4-b60f-9f040aecb380 admin admin] PATH_INFO: `/v3/projects/3def9cdd433645ddb213c43a5a38ff82/users/fd390f43a3f94618835b99488eee8bcf/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.157457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1315/2657] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:33 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:33.158862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a73fac-12e3-479b-aebe-3be22b59af76 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.163768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f96cb-39fa-44b4-b60f-9f040aecb380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3def9cdd433645ddb213c43a5a38ff82, user_id=fd390f43a3f94618835b99488eee8bcf, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.166635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f96cb-39fa-44b4-b60f-9f040aecb380 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.168950 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a73fac-12e3-479b-aebe-3be22b59af76 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.173814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd532e5-b684-48e6-849c-805e266ad04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.174506 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd532e5-b684-48e6-849c-805e266ad04e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.174881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd532e5-b684-48e6-849c-805e266ad04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.175180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd532e5-b684-48e6-849c-805e266ad04e admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.175908 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd532e5-b684-48e6-849c-805e266ad04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.177570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd532e5-b684-48e6-849c-805e266ad04e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.182569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1343/2658] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:33 2022] PUT /identity/v3/projects/3def9cdd433645ddb213c43a5a38ff82/users/fd390f43a3f94618835b99488eee8bcf/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:33.189947 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1316/2659] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:33.192305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f96cb-39fa-44b4-b60f-9f040aecb380 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.192684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f157ea2-3000-49bd-977d-f181e3c23c60 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.192983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f157ea2-3000-49bd-977d-f181e3c23c60 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.193304 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f157ea2-3000-49bd-977d-f181e3c23c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.220363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6f157ea2-3000-49bd-977d-f181e3c23c60 None None] MFA Rules not processed for user `fd390f43a3f94618835b99488eee8bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:33.221662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6f157ea2-3000-49bd-977d-f181e3c23c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:33.235614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:33.266279 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f96cb-39fa-44b4-b60f-9f040aecb380 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.273968 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d941e-1611-4531-aead-d761deeb7aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.275603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d941e-1611-4531-aead-d761deeb7aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.275946 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d941e-1611-4531-aead-d761deeb7aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.276381 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d941e-1611-4531-aead-d761deeb7aa2 admin admin] PATH_INFO: `/v3/projects/207b83a6e015404bbf712899279fccd4/users/8da23b74540a4006831eddc02bd0c198/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.299010 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d941e-1611-4531-aead-d761deeb7aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=207b83a6e015404bbf712899279fccd4, user_id=8da23b74540a4006831eddc02bd0c198, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.303842 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d941e-1611-4531-aead-d761deeb7aa2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.312040 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1317/2660] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:33.318029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f157ea2-3000-49bd-977d-f181e3c23c60 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.320650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1344/2661] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:33 2022] PUT /identity/v3/projects/207b83a6e015404bbf712899279fccd4/users/8da23b74540a4006831eddc02bd0c198/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:33.325055 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d941e-1611-4531-aead-d761deeb7aa2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.332855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f157ea2-3000-49bd-977d-f181e3c23c60 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.362831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:33.393058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d941e-1611-4531-aead-d761deeb7aa2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.395423 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245879c1-28c6-4b32-8679-47f47501577e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.396112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245879c1-28c6-4b32-8679-47f47501577e admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.396447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245879c1-28c6-4b32-8679-47f47501577e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.396781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245879c1-28c6-4b32-8679-47f47501577e admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.397429 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245879c1-28c6-4b32-8679-47f47501577e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.397731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15884b9-b0de-49d1-961b-27999829e8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.398399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15884b9-b0de-49d1-961b-27999829e8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.398679 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15884b9-b0de-49d1-961b-27999829e8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.399013 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15884b9-b0de-49d1-961b-27999829e8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.399422 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245879c1-28c6-4b32-8679-47f47501577e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.399747 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15884b9-b0de-49d1-961b-27999829e8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.401428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15884b9-b0de-49d1-961b-27999829e8a3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.411538 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1345/2662] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:33.415621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15884b9-b0de-49d1-961b-27999829e8a3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.426085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15884b9-b0de-49d1-961b-27999829e8a3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.426812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1318/2663] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:33 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:33.430675 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245879c1-28c6-4b32-8679-47f47501577e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.490409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1bcdf-9ddb-438f-9fa8-4b3db1187360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.491332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1bcdf-9ddb-438f-9fa8-4b3db1187360 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.491612 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1bcdf-9ddb-438f-9fa8-4b3db1187360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.491879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1bcdf-9ddb-438f-9fa8-4b3db1187360 admin admin] PATH_INFO: `/v3/projects/207b83a6e015404bbf712899279fccd4/users/8da23b74540a4006831eddc02bd0c198/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.493852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1bcdf-9ddb-438f-9fa8-4b3db1187360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=207b83a6e015404bbf712899279fccd4, user_id=8da23b74540a4006831eddc02bd0c198, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.496583 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1bcdf-9ddb-438f-9fa8-4b3db1187360 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.499909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245879c1-28c6-4b32-8679-47f47501577e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.504931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355fb4f-083d-4146-8ae7-fb848b7944be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.505604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355fb4f-083d-4146-8ae7-fb848b7944be admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.505889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355fb4f-083d-4146-8ae7-fb848b7944be admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.506164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355fb4f-083d-4146-8ae7-fb848b7944be admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.506916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355fb4f-083d-4146-8ae7-fb848b7944be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.507170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1346/2664] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:33 2022] PUT /identity/v3/projects/207b83a6e015404bbf712899279fccd4/users/8da23b74540a4006831eddc02bd0c198/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:33.508831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355fb4f-083d-4146-8ae7-fb848b7944be admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.517642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e45cf-6d59-4a8b-a15c-3f05bc433fb2 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.517930 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e45cf-6d59-4a8b-a15c-3f05bc433fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.518201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e45cf-6d59-4a8b-a15c-3f05bc433fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.534736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1319/2665] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:33 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:33.540973 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355fb4f-083d-4146-8ae7-fb848b7944be admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.546970 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355fb4f-083d-4146-8ae7-fb848b7944be admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.550919 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-911e45cf-6d59-4a8b-a15c-3f05bc433fb2 None None] MFA Rules not processed for user `8da23b74540a4006831eddc02bd0c198`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:33.552309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-911e45cf-6d59-4a8b-a15c-3f05bc433fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:33.594491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:33.605894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dee91-5e9d-4fd3-a478-8e334a64209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.606783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dee91-5e9d-4fd3-a478-8e334a64209e admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.607134 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dee91-5e9d-4fd3-a478-8e334a64209e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.607507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dee91-5e9d-4fd3-a478-8e334a64209e admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.608364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31dee91-5e9d-4fd3-a478-8e334a64209e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.610239 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31dee91-5e9d-4fd3-a478-8e334a64209e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.620501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1320/2666] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:49:33.624259 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1347/2667] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:33.627741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dee91-5e9d-4fd3-a478-8e334a64209e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.635044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e45cf-6d59-4a8b-a15c-3f05bc433fb2 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.636468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-911e45cf-6d59-4a8b-a15c-3f05bc433fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:33.637509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dee91-5e9d-4fd3-a478-8e334a64209e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.642092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7acc0d-36e4-466d-af12-492bb51c9b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.643774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7acc0d-36e4-466d-af12-492bb51c9b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.644072 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7acc0d-36e4-466d-af12-492bb51c9b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.644351 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7acc0d-36e4-466d-af12-492bb51c9b71 admin admin] PATH_INFO: `/v3/projects/ba34d2ca2c0048298d008fa43596099e/users/67bb6793b48a423996efdfe3c09d3b22/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.659817 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7acc0d-36e4-466d-af12-492bb51c9b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba34d2ca2c0048298d008fa43596099e, user_id=67bb6793b48a423996efdfe3c09d3b22, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.662419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7acc0d-36e4-466d-af12-492bb51c9b71 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.679060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1321/2668] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:33 2022] PUT /identity/v3/projects/ba34d2ca2c0048298d008fa43596099e/users/67bb6793b48a423996efdfe3c09d3b22/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:33.683114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7acc0d-36e4-466d-af12-492bb51c9b71 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.695995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e45cf-6d59-4a8b-a15c-3f05bc433fb2 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.697343 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:33.748469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7886c-9126-4d7f-b039-2f31fb6572e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.749153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7886c-9126-4d7f-b039-2f31fb6572e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.749463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7886c-9126-4d7f-b039-2f31fb6572e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.749745 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7886c-9126-4d7f-b039-2f31fb6572e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.750408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7acc0d-36e4-466d-af12-492bb51c9b71 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.755938 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c13705-8fe3-48bf-8cfb-8042442b2d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.756607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c13705-8fe3-48bf-8cfb-8042442b2d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.756892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c13705-8fe3-48bf-8cfb-8042442b2d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.757197 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c13705-8fe3-48bf-8cfb-8042442b2d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.758009 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c13705-8fe3-48bf-8cfb-8042442b2d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.759607 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c13705-8fe3-48bf-8cfb-8042442b2d2b admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.773221 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1322/2669] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:33.774461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c13705-8fe3-48bf-8cfb-8042442b2d2b admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.784896 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c13705-8fe3-48bf-8cfb-8042442b2d2b admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.789512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9856a7f-2c4a-4a82-a2e8-b5033423a369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:33.790382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856a7f-2c4a-4a82-a2e8-b5033423a369 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.790656 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856a7f-2c4a-4a82-a2e8-b5033423a369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.791098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9856a7f-2c4a-4a82-a2e8-b5033423a369 admin admin] PATH_INFO: `/v3/projects/ba34d2ca2c0048298d008fa43596099e/users/67bb6793b48a423996efdfe3c09d3b22/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.793202 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9856a7f-2c4a-4a82-a2e8-b5033423a369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba34d2ca2c0048298d008fa43596099e, user_id=67bb6793b48a423996efdfe3c09d3b22, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.795862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9856a7f-2c4a-4a82-a2e8-b5033423a369 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.806179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1323/2670] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:33 2022] PUT /identity/v3/projects/ba34d2ca2c0048298d008fa43596099e/users/67bb6793b48a423996efdfe3c09d3b22/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:33.814397 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7886c-9126-4d7f-b039-2f31fb6572e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:33.816094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7886c-9126-4d7f-b039-2f31fb6572e8 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:33.817496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb2ad1-91e9-4793-9c6c-92429d2c53e2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:33.817795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb2ad1-91e9-4793-9c6c-92429d2c53e2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:33.818078 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb2ad1-91e9-4793-9c6c-92429d2c53e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:33.827012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1348/2671] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:49:33.852364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-46fb2ad1-91e9-4793-9c6c-92429d2c53e2 None None] MFA Rules not processed for user `67bb6793b48a423996efdfe3c09d3b22`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:33.853667 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-46fb2ad1-91e9-4793-9c6c-92429d2c53e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:33.925795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1324/2672] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:33.938508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7886c-9126-4d7f-b039-2f31fb6572e8 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:33.940080 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ddf7886c-9126-4d7f-b039-2f31fb6572e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:33.951757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7886c-9126-4d7f-b039-2f31fb6572e8 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:33.995230 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=100 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:34.006309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600e4aa-d03d-4b70-a7b3-ec10177029a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:34.007039 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600e4aa-d03d-4b70-a7b3-ec10177029a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:34.007302 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600e4aa-d03d-4b70-a7b3-ec10177029a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:34.007622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600e4aa-d03d-4b70-a7b3-ec10177029a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:34.017393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600e4aa-d03d-4b70-a7b3-ec10177029a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:34.018934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600e4aa-d03d-4b70-a7b3-ec10177029a6 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:34.028999 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1349/2673] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:34.106573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb2ad1-91e9-4793-9c6c-92429d2c53e2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:34.116614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb2ad1-91e9-4793-9c6c-92429d2c53e2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:34.121154 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40b1ec-0d9e-4610-a78b-427da8a3899c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:34.121819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40b1ec-0d9e-4610-a78b-427da8a3899c service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:34.122115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40b1ec-0d9e-4610-a78b-427da8a3899c service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:34.122382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40b1ec-0d9e-4610-a78b-427da8a3899c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:34.185708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40b1ec-0d9e-4610-a78b-427da8a3899c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:34.187125 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40b1ec-0d9e-4610-a78b-427da8a3899c service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:34.196486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1325/2674] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:34.361642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600e4aa-d03d-4b70-a7b3-ec10177029a6 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:34.370452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600e4aa-d03d-4b70-a7b3-ec10177029a6 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:34.374943 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff0867-a97d-46b6-9cd5-cf501f3163d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:34.375690 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0867-a97d-46b6-9cd5-cf501f3163d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:34.375977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0867-a97d-46b6-9cd5-cf501f3163d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:34.376255 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff0867-a97d-46b6-9cd5-cf501f3163d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:34.381100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-edff0867-a97d-46b6-9cd5-cf501f3163d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:34.444599 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff0867-a97d-46b6-9cd5-cf501f3163d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:34.445972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff0867-a97d-46b6-9cd5-cf501f3163d1 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:34.456539 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1350/2675] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:34.466648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40b1ec-0d9e-4610-a78b-427da8a3899c service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:34.487961 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:34.533802 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40b1ec-0d9e-4610-a78b-427da8a3899c service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:34.539326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:34.540218 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:34.540494 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:34.540778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] PATH_INFO: `/v3/users/fd390f43a3f94618835b99488eee8bcf` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:34.542271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd390f43a3f94618835b99488eee8bcf)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:34.544274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:34.594628 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd390f43a3f94618835b99488eee8bcf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:34.595374 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd390f43a3f94618835b99488eee8bcf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:34.608837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd390f43a3f94618835b99488eee8bcf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:34.609641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd390f43a3f94618835b99488eee8bcf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:34.618091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd390f43a3f94618835b99488eee8bcf'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:34.640588 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1326/2676] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:34 2022] DELETE /identity/v3/users/fd390f43a3f94618835b99488eee8bcf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:34.645969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c1ea-6f6a-4b47-ba9f-198b16a4b5b1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:34.646262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c1ea-6f6a-4b47-ba9f-198b16a4b5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:34.646540 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c1ea-6f6a-4b47-ba9f-198b16a4b5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:34.668608 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-fac9c1ea-6f6a-4b47-ba9f-198b16a4b5b1 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:34.669918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fac9c1ea-6f6a-4b47-ba9f-198b16a4b5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:34.738052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1351/2677] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:34.748844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:34.752958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:34.816160 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c11fb-ed2e-48e2-a801-71727c6bdd27 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:34.820305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5869c1-148d-47db-9712-5521b6302d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:34.821007 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5869c1-148d-47db-9712-5521b6302d82 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:34.821318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5869c1-148d-47db-9712-5521b6302d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:34.821634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5869c1-148d-47db-9712-5521b6302d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:34.832110 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5869c1-148d-47db-9712-5521b6302d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:34.833559 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5869c1-148d-47db-9712-5521b6302d82 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:34.844229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1327/2678] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:35.127899 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9c1ea-6f6a-4b47-ba9f-198b16a4b5b1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:35.129522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-fac9c1ea-6f6a-4b47-ba9f-198b16a4b5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:35.144239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9c1ea-6f6a-4b47-ba9f-198b16a4b5b1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:35.149936 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:35.150820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:35.151074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:35.151392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] PATH_INFO: `/v3/projects/3def9cdd433645ddb213c43a5a38ff82` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:35.152897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3def9cdd433645ddb213c43a5a38ff82)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:35.155319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:35.205379 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3def9cdd433645ddb213c43a5a38ff82', 'request_id': 'req-6c79aba4-1cda-49f6-9186-35bb64896d09'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.211489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3def9cdd433645ddb213c43a5a38ff82', 'request_id': 'req-6c79aba4-1cda-49f6-9186-35bb64896d09'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.217572 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3def9cdd433645ddb213c43a5a38ff82', 'request_id': 'req-6c79aba4-1cda-49f6-9186-35bb64896d09'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.236533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] The token cache is being invalidate because project 3def9cdd433645ddb213c43a5a38ff82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:35.236758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.237533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.239219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1352/2679] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:35 2022] DELETE /identity/v3/projects/3def9cdd433645ddb213c43a5a38ff82 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:35.244435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5869c1-148d-47db-9712-5521b6302d82 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:35.254305 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5869c1-148d-47db-9712-5521b6302d82 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:35.295616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:35.314028 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcff005-8801-4634-8113-0a2b8655eb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:35.314940 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcff005-8801-4634-8113-0a2b8655eb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:35.315223 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcff005-8801-4634-8113-0a2b8655eb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:35.315496 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcff005-8801-4634-8113-0a2b8655eb14 admin admin] PATH_INFO: `/v3/users/67bb6793b48a423996efdfe3c09d3b22` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:35.316691 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcff005-8801-4634-8113-0a2b8655eb14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67bb6793b48a423996efdfe3c09d3b22)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:35.318795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcff005-8801-4634-8113-0a2b8655eb14 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:35.367192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3fcff005-8801-4634-8113-0a2b8655eb14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67bb6793b48a423996efdfe3c09d3b22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.367849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3fcff005-8801-4634-8113-0a2b8655eb14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67bb6793b48a423996efdfe3c09d3b22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.380763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3fcff005-8801-4634-8113-0a2b8655eb14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67bb6793b48a423996efdfe3c09d3b22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.381404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3fcff005-8801-4634-8113-0a2b8655eb14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67bb6793b48a423996efdfe3c09d3b22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.392443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3fcff005-8801-4634-8113-0a2b8655eb14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67bb6793b48a423996efdfe3c09d3b22'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.404385 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1328/2680] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:35 2022] DELETE /identity/v3/users/67bb6793b48a423996efdfe3c09d3b22 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:35.693046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:35.695180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:35.711043 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79aba4-1cda-49f6-9186-35bb64896d09 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:35.716819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:35.782932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:35.783810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:35.784089 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:35.784363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] PATH_INFO: `/v3/projects/ba34d2ca2c0048298d008fa43596099e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:35.785576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba34d2ca2c0048298d008fa43596099e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:35.787560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:35.837326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba34d2ca2c0048298d008fa43596099e', 'request_id': 'req-091c680e-f08f-416f-81c1-edd2b30aff93'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.844616 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba34d2ca2c0048298d008fa43596099e', 'request_id': 'req-091c680e-f08f-416f-81c1-edd2b30aff93'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.850770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba34d2ca2c0048298d008fa43596099e', 'request_id': 'req-091c680e-f08f-416f-81c1-edd2b30aff93'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.864562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] The token cache is being invalidate because project ba34d2ca2c0048298d008fa43596099e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:35.864858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.865578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-091c680e-f08f-416f-81c1-edd2b30aff93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:35.867309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1353/2681] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:35 2022] DELETE /identity/v3/projects/ba34d2ca2c0048298d008fa43596099e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:35.873883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f6f9a-515b-4283-a086-eb68f31c2986 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:35.874185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f6f9a-515b-4283-a086-eb68f31c2986 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:35.874460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f6f9a-515b-4283-a086-eb68f31c2986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:35.895444 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c22f6f9a-515b-4283-a086-eb68f31c2986 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:35.896721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c22f6f9a-515b-4283-a086-eb68f31c2986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:35.963340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1329/2682] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:35.973666 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139ce13-f680-45ab-9303-83ff6ac6015f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:35.974124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139ce13-f680-45ab-9303-83ff6ac6015f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:35.974402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139ce13-f680-45ab-9303-83ff6ac6015f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:35.995477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c139ce13-f680-45ab-9303-83ff6ac6015f None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:35.996804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c139ce13-f680-45ab-9303-83ff6ac6015f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:36.004932 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1354/2683] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:36.009511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f6f9a-515b-4283-a086-eb68f31c2986 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:36.019619 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f6f9a-515b-4283-a086-eb68f31c2986 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:36.024694 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700972b9-e094-4d41-a4c6-7d81a40f0089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:36.025352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700972b9-e094-4d41-a4c6-7d81a40f0089 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:36.025635 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700972b9-e094-4d41-a4c6-7d81a40f0089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:36.025896 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700972b9-e094-4d41-a4c6-7d81a40f0089 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:36.026656 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700972b9-e094-4d41-a4c6-7d81a40f0089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:36.027002 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700972b9-e094-4d41-a4c6-7d81a40f0089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:36.029013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700972b9-e094-4d41-a4c6-7d81a40f0089 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:36.041356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1330/2684] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:49:36.046438 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139ce13-f680-45ab-9303-83ff6ac6015f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:36.056324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139ce13-f680-45ab-9303-83ff6ac6015f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:36.062279 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03945afd-143f-43fb-87cc-c1c48325ae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:36.062997 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03945afd-143f-43fb-87cc-c1c48325ae71 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:36.063281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03945afd-143f-43fb-87cc-c1c48325ae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:36.063554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03945afd-143f-43fb-87cc-c1c48325ae71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:36.064253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03945afd-143f-43fb-87cc-c1c48325ae71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:36.066236 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03945afd-143f-43fb-87cc-c1c48325ae71 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:36.095553 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1355/2685] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:36 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:36.099896 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700972b9-e094-4d41-a4c6-7d81a40f0089 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:36.110234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700972b9-e094-4d41-a4c6-7d81a40f0089 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:36.154484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:36.168690 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65199b63-244b-4da3-988b-bd4f0fc738ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:36.169404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65199b63-244b-4da3-988b-bd4f0fc738ea admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:36.169689 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65199b63-244b-4da3-988b-bd4f0fc738ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:36.169964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65199b63-244b-4da3-988b-bd4f0fc738ea admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:36.170833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65199b63-244b-4da3-988b-bd4f0fc738ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:36.172800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65199b63-244b-4da3-988b-bd4f0fc738ea admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:36.194865 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1331/2686] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:36 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:36.199194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03945afd-143f-43fb-87cc-c1c48325ae71 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:36.210186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03945afd-143f-43fb-87cc-c1c48325ae71 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:36.217793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08411afb-579c-4b9e-a67e-b6b944efe5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:36.218531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08411afb-579c-4b9e-a67e-b6b944efe5bd admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:36.218859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08411afb-579c-4b9e-a67e-b6b944efe5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:36.219190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08411afb-579c-4b9e-a67e-b6b944efe5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:36.219972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08411afb-579c-4b9e-a67e-b6b944efe5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:36.221705 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08411afb-579c-4b9e-a67e-b6b944efe5bd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:36.233319 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1356/2687] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:36.237491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65199b63-244b-4da3-988b-bd4f0fc738ea admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:36.249846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65199b63-244b-4da3-988b-bd4f0fc738ea admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:36.255956 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608954b-dc17-4792-8be2-4ed4359df0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:36.256727 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608954b-dc17-4792-8be2-4ed4359df0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:36.257051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608954b-dc17-4792-8be2-4ed4359df0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:36.257351 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608954b-dc17-4792-8be2-4ed4359df0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:36.258236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608954b-dc17-4792-8be2-4ed4359df0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:36.259989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608954b-dc17-4792-8be2-4ed4359df0e1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:36.271682 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1332/2688] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:36.275976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08411afb-579c-4b9e-a67e-b6b944efe5bd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:36.286061 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08411afb-579c-4b9e-a67e-b6b944efe5bd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:36.290966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6086de-fa68-405b-8442-fbff43bab16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:36.291852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6086de-fa68-405b-8442-fbff43bab16a admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:36.292134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6086de-fa68-405b-8442-fbff43bab16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:36.292492 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6086de-fa68-405b-8442-fbff43bab16a admin admin] PATH_INFO: `/v3/projects/2e52b91b186f45e5846c46131d185e81/users/1c0a8a42dd0d4f659c87c661095399de/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:36.309109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6086de-fa68-405b-8442-fbff43bab16a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e52b91b186f45e5846c46131d185e81, user_id=1c0a8a42dd0d4f659c87c661095399de, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:36.312087 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6086de-fa68-405b-8442-fbff43bab16a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:36.326152 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1357/2689] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:36 2022] PUT /identity/v3/projects/2e52b91b186f45e5846c46131d185e81/users/1c0a8a42dd0d4f659c87c661095399de/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:36.336747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e7bad-6f5e-4381-8b97-6ace4f1ea929 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:36.337043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e7bad-6f5e-4381-8b97-6ace4f1ea929 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:36.337368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e7bad-6f5e-4381-8b97-6ace4f1ea929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:36.368994 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-698e7bad-6f5e-4381-8b97-6ace4f1ea929 None None] MFA Rules not processed for user `1c0a8a42dd0d4f659c87c661095399de`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:36.370234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-698e7bad-6f5e-4381-8b97-6ace4f1ea929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:36.439815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1333/2690] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:36.451075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6086de-fa68-405b-8442-fbff43bab16a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:36.454233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-ee6086de-fa68-405b-8442-fbff43bab16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:36.490965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:36.518651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6086de-fa68-405b-8442-fbff43bab16a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:36.522720 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc7167-5b79-4971-b1d1-3ff3fd1da4a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:36.523402 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc7167-5b79-4971-b1d1-3ff3fd1da4a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:36.523689 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc7167-5b79-4971-b1d1-3ff3fd1da4a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:36.523969 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc7167-5b79-4971-b1d1-3ff3fd1da4a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:36.534657 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc7167-5b79-4971-b1d1-3ff3fd1da4a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:36.536986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdc7167-5b79-4971-b1d1-3ff3fd1da4a0 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:36.548807 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1358/2691] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:38.511914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e7bad-6f5e-4381-8b97-6ace4f1ea929 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:38.513474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-698e7bad-6f5e-4381-8b97-6ace4f1ea929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:38.526513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e7bad-6f5e-4381-8b97-6ace4f1ea929 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:38.530250 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645bf7b0-3e50-40de-9d43-4777bfbc551d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:38.530971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bf7b0-3e50-40de-9d43-4777bfbc551d service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:38.531259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bf7b0-3e50-40de-9d43-4777bfbc551d service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:38.531527 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bf7b0-3e50-40de-9d43-4777bfbc551d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:38.533605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-645bf7b0-3e50-40de-9d43-4777bfbc551d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:38.597539 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bf7b0-3e50-40de-9d43-4777bfbc551d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:38.599011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bf7b0-3e50-40de-9d43-4777bfbc551d service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:38.608236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1334/2692] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:39.579636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc7167-5b79-4971-b1d1-3ff3fd1da4a0 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:39.643812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc7167-5b79-4971-b1d1-3ff3fd1da4a0 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:39.648449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:39.649327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:39.649732 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:39.650003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] PATH_INFO: `/v3/users/1c0a8a42dd0d4f659c87c661095399de` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:39.651260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c0a8a42dd0d4f659c87c661095399de)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:39.653271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:39.723742 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c0a8a42dd0d4f659c87c661095399de'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:39.724584 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c0a8a42dd0d4f659c87c661095399de'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:39.739365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c0a8a42dd0d4f659c87c661095399de'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:39.740070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c0a8a42dd0d4f659c87c661095399de'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:39.749312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c0a8a42dd0d4f659c87c661095399de'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:39.763637 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1359/2693] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:39 2022] DELETE /identity/v3/users/1c0a8a42dd0d4f659c87c661095399de => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:39.769288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c6615-6ac3-4cf0-9017-faa2bd3b0831 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:39.769783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c6615-6ac3-4cf0-9017-faa2bd3b0831 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:39.769900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c6615-6ac3-4cf0-9017-faa2bd3b0831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:39.789504 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-400c6615-6ac3-4cf0-9017-faa2bd3b0831 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:39.790731 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-400c6615-6ac3-4cf0-9017-faa2bd3b0831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:39.847661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:39.861440 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1335/2694] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:39.871246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:39.872783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:39.932120 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819e9f7-bb48-4f6f-9bc9-d43e21ff8d32 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:39.936179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb250a-72d4-4452-993c-84714d974625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:39.936826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb250a-72d4-4452-993c-84714d974625 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:39.937105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb250a-72d4-4452-993c-84714d974625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:39.937380 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb250a-72d4-4452-993c-84714d974625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:39.947312 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb250a-72d4-4452-993c-84714d974625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:39.948780 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb250a-72d4-4452-993c-84714d974625 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:39.952740 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:39.960134 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1360/2695] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:40.267538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c6615-6ac3-4cf0-9017-faa2bd3b0831 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:40.269097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-400c6615-6ac3-4cf0-9017-faa2bd3b0831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:40.281253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c6615-6ac3-4cf0-9017-faa2bd3b0831 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:40.288708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:40.289597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:40.289875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:40.290144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] PATH_INFO: `/v3/projects/2e52b91b186f45e5846c46131d185e81` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:40.291359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e52b91b186f45e5846c46131d185e81)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:40.293274 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:40.345810 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e52b91b186f45e5846c46131d185e81', 'request_id': 'req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:40.352320 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e52b91b186f45e5846c46131d185e81', 'request_id': 'req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:40.358967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e52b91b186f45e5846c46131d185e81', 'request_id': 'req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:40.373838 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] The token cache is being invalidate because project 2e52b91b186f45e5846c46131d185e81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:40.374052 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:40.374912 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-25a1d190-eb7d-4eca-93ca-0adcdadd14f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:40.376743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1336/2696] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:40 2022] DELETE /identity/v3/projects/2e52b91b186f45e5846c46131d185e81 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:40.810298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb250a-72d4-4452-993c-84714d974625 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:40.812003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-aaeb250a-72d4-4452-993c-84714d974625 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:40.826153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb250a-72d4-4452-993c-84714d974625 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:40.886545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:40.887454 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:40.887734 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:40.888008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] PATH_INFO: `/v3/users/2f6be4677f1c441c8df4d079a1f38679` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:40.889273 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f6be4677f1c441c8df4d079a1f38679)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:40.891399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:40.941280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f6be4677f1c441c8df4d079a1f38679'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:40.941858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f6be4677f1c441c8df4d079a1f38679'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:40.954171 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f6be4677f1c441c8df4d079a1f38679'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:40.954976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f6be4677f1c441c8df4d079a1f38679'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:40.964830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f6be4677f1c441c8df4d079a1f38679'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:40.977448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1361/2697] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:40 2022] DELETE /identity/v3/users/2f6be4677f1c441c8df4d079a1f38679 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:40.982102 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d1205-1dba-411c-92a4-a5fad3788d4c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:40.982600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d1205-1dba-411c-92a4-a5fad3788d4c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:40.982718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d1205-1dba-411c-92a4-a5fad3788d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:41.000390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f04d1205-1dba-411c-92a4-a5fad3788d4c None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:41.001590 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f04d1205-1dba-411c-92a4-a5fad3788d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:41.067353 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1337/2698] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:41.078876 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:41.080471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:41.143046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4b223-e131-4d1e-8aa5-0b9f65d743d2 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:41.146858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f8dc5-4863-4691-be02-c01a8b405ff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:41.147504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f8dc5-4863-4691-be02-c01a8b405ff0 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:41.147782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f8dc5-4863-4691-be02-c01a8b405ff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:41.148053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f8dc5-4863-4691-be02-c01a8b405ff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:41.165542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f8dc5-4863-4691-be02-c01a8b405ff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:41.167795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f8dc5-4863-4691-be02-c01a8b405ff0 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:41.180278 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1362/2699] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:41.477033 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d1205-1dba-411c-92a4-a5fad3788d4c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:41.478844 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f04d1205-1dba-411c-92a4-a5fad3788d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:41.492550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d1205-1dba-411c-92a4-a5fad3788d4c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:41.498616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:41.499638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:41.500121 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:41.500509 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] PATH_INFO: `/v3/projects/7667209b909a4f56a89d1be06ced4431` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:41.501931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7667209b909a4f56a89d1be06ced4431)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:41.503944 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:41.553307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7667209b909a4f56a89d1be06ced4431', 'request_id': 'req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:41.559800 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7667209b909a4f56a89d1be06ced4431', 'request_id': 'req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:41.565897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7667209b909a4f56a89d1be06ced4431', 'request_id': 'req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:41.581159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] The token cache is being invalidate because project 7667209b909a4f56a89d1be06ced4431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:41.581450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:41.582115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:41.583750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1338/2700] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:41 2022] DELETE /identity/v3/projects/7667209b909a4f56a89d1be06ced4431 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:49:41.588003 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f8dc5-4863-4691-be02-c01a8b405ff0 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:41.598292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f8dc5-4863-4691-be02-c01a8b405ff0 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:41.615806 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:41.657792 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:41.658679 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:41.658990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:41.659264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] PATH_INFO: `/v3/users/eefe37811d7d424baac016d6ba959316` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:41.660474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eefe37811d7d424baac016d6ba959316)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:41.662419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:41.714185 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eefe37811d7d424baac016d6ba959316'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:41.714960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eefe37811d7d424baac016d6ba959316'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:41.729225 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eefe37811d7d424baac016d6ba959316'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:41.729971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eefe37811d7d424baac016d6ba959316'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:41.740056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eefe37811d7d424baac016d6ba959316'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:41.757859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1363/2701] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:41 2022] DELETE /identity/v3/users/eefe37811d7d424baac016d6ba959316 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:42.047786 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:42.049460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:42.057792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:42.063984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97c66a-28fc-4572-b5b6-c58ada1ac3e1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:42.122641 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:42.123572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:42.123858 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:42.124131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] PATH_INFO: `/v3/projects/a7567ebed2d2448a8d5dfdbfd9486f7c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:42.125457 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7567ebed2d2448a8d5dfdbfd9486f7c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:42.127461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:42.177891 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7567ebed2d2448a8d5dfdbfd9486f7c', 'request_id': 'req-47a612db-ba3d-4a36-91a4-67a065b1a2f7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.184652 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7567ebed2d2448a8d5dfdbfd9486f7c', 'request_id': 'req-47a612db-ba3d-4a36-91a4-67a065b1a2f7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.190914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7567ebed2d2448a8d5dfdbfd9486f7c', 'request_id': 'req-47a612db-ba3d-4a36-91a4-67a065b1a2f7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.206261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] The token cache is being invalidate because project a7567ebed2d2448a8d5dfdbfd9486f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:42.206547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.207222 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.208897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1339/2702] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:42 2022] DELETE /identity/v3/projects/a7567ebed2d2448a8d5dfdbfd9486f7c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:42.214814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:42.216428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:42.230368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6f772-fa75-429d-b849-2b9682113a56 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:42.293396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59f69d-1d17-4e42-88b5-33aaf25b2759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:42.294203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59f69d-1d17-4e42-88b5-33aaf25b2759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:42.294480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59f69d-1d17-4e42-88b5-33aaf25b2759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:42.294856 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59f69d-1d17-4e42-88b5-33aaf25b2759 admin admin] PATH_INFO: `/v3/users/8da23b74540a4006831eddc02bd0c198` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:42.296068 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59f69d-1d17-4e42-88b5-33aaf25b2759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8da23b74540a4006831eddc02bd0c198)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:42.298046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59f69d-1d17-4e42-88b5-33aaf25b2759 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:42.345624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:42.351736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5b59f69d-1d17-4e42-88b5-33aaf25b2759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8da23b74540a4006831eddc02bd0c198'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.352618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5b59f69d-1d17-4e42-88b5-33aaf25b2759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8da23b74540a4006831eddc02bd0c198'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.367116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5b59f69d-1d17-4e42-88b5-33aaf25b2759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8da23b74540a4006831eddc02bd0c198'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.367816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5b59f69d-1d17-4e42-88b5-33aaf25b2759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8da23b74540a4006831eddc02bd0c198'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.376486 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-5b59f69d-1d17-4e42-88b5-33aaf25b2759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8da23b74540a4006831eddc02bd0c198'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.388484 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1364/2703] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:42 2022] DELETE /identity/v3/users/8da23b74540a4006831eddc02bd0c198 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:42.670362 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:42.672759 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:42.690141 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a612db-ba3d-4a36-91a4-67a065b1a2f7 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:42.749785 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:42.750739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:42.751206 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:42.751632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] PATH_INFO: `/v3/projects/207b83a6e015404bbf712899279fccd4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:42.752981 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=207b83a6e015404bbf712899279fccd4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:42.754907 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:42.799390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:42.810499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '207b83a6e015404bbf712899279fccd4', 'request_id': 'req-997078e3-9e89-4ad8-92e7-408ea20595a2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.817126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '207b83a6e015404bbf712899279fccd4', 'request_id': 'req-997078e3-9e89-4ad8-92e7-408ea20595a2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.822739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '207b83a6e015404bbf712899279fccd4', 'request_id': 'req-997078e3-9e89-4ad8-92e7-408ea20595a2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.837122 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] The token cache is being invalidate because project 207b83a6e015404bbf712899279fccd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:42.837408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.838007 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-997078e3-9e89-4ad8-92e7-408ea20595a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:42.840202 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1340/2704] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:42 2022] DELETE /identity/v3/projects/207b83a6e015404bbf712899279fccd4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:42.846256 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e8117-981d-46f6-95bc-e4cb8536b30a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:42.846542 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e8117-981d-46f6-95bc-e4cb8536b30a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:42.846849 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e8117-981d-46f6-95bc-e4cb8536b30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:42.872293 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-284e8117-981d-46f6-95bc-e4cb8536b30a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:42.874153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-284e8117-981d-46f6-95bc-e4cb8536b30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:42.936775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1365/2705] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:42.945768 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7651cd3-9deb-4519-a89e-b809771306d5 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:42.946151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7651cd3-9deb-4519-a89e-b809771306d5 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:42.946431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7651cd3-9deb-4519-a89e-b809771306d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:42.966465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a7651cd3-9deb-4519-a89e-b809771306d5 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:42.967713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a7651cd3-9deb-4519-a89e-b809771306d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:42.976510 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1341/2706] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:42.980897 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e8117-981d-46f6-95bc-e4cb8536b30a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:42.991764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e8117-981d-46f6-95bc-e4cb8536b30a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:42.997400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c335a86-e434-4b20-a4e2-f8368b79ce83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:42.998064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c335a86-e434-4b20-a4e2-f8368b79ce83 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:42.998347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c335a86-e434-4b20-a4e2-f8368b79ce83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:42.998612 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c335a86-e434-4b20-a4e2-f8368b79ce83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:42.999354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c335a86-e434-4b20-a4e2-f8368b79ce83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:42.999662 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c335a86-e434-4b20-a4e2-f8368b79ce83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.001504 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c335a86-e434-4b20-a4e2-f8368b79ce83 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.013621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1366/2707] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:49:43.017864 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7651cd3-9deb-4519-a89e-b809771306d5 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:43.027370 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7651cd3-9deb-4519-a89e-b809771306d5 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:43.033313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b3176-399e-41c3-b628-65d4daa2a588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:43.033983 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b3176-399e-41c3-b628-65d4daa2a588 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.034264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b3176-399e-41c3-b628-65d4daa2a588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.034537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b3176-399e-41c3-b628-65d4daa2a588 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.035285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b3176-399e-41c3-b628-65d4daa2a588 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.037204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b3176-399e-41c3-b628-65d4daa2a588 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.067452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1342/2708] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:43 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:43.071887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c335a86-e434-4b20-a4e2-f8368b79ce83 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:43.082566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c335a86-e434-4b20-a4e2-f8368b79ce83 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:43.143954 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8bd00-8daf-4d34-a000-2310474534f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:43.144624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8bd00-8daf-4d34-a000-2310474534f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.144903 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8bd00-8daf-4d34-a000-2310474534f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.145179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8bd00-8daf-4d34-a000-2310474534f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.145869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8bd00-8daf-4d34-a000-2310474534f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.147834 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8bd00-8daf-4d34-a000-2310474534f8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.169904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1367/2709] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:43 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:43.175863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b3176-399e-41c3-b628-65d4daa2a588 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:43.192185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b3176-399e-41c3-b628-65d4daa2a588 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:43.200077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837bb39-eb0f-4b89-a63a-ae099b7c38be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:43.200949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837bb39-eb0f-4b89-a63a-ae099b7c38be admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.201232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837bb39-eb0f-4b89-a63a-ae099b7c38be admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.201557 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837bb39-eb0f-4b89-a63a-ae099b7c38be admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.202620 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837bb39-eb0f-4b89-a63a-ae099b7c38be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.204565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837bb39-eb0f-4b89-a63a-ae099b7c38be admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.216443 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1343/2710] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:43.221218 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8bd00-8daf-4d34-a000-2310474534f8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:43.231617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8bd00-8daf-4d34-a000-2310474534f8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:43.237149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc8da7-0a3d-4598-a43d-e6db1b805f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:43.237794 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc8da7-0a3d-4598-a43d-e6db1b805f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.238076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc8da7-0a3d-4598-a43d-e6db1b805f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.238355 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc8da7-0a3d-4598-a43d-e6db1b805f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.239128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc8da7-0a3d-4598-a43d-e6db1b805f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.240850 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc8da7-0a3d-4598-a43d-e6db1b805f5f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.251067 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1368/2711] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:43.255242 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837bb39-eb0f-4b89-a63a-ae099b7c38be admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:43.265691 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837bb39-eb0f-4b89-a63a-ae099b7c38be admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:43.270587 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3542ab1-d495-419e-96f7-426a1460b7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:43.271501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3542ab1-d495-419e-96f7-426a1460b7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.271781 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3542ab1-d495-419e-96f7-426a1460b7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.272048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3542ab1-d495-419e-96f7-426a1460b7f5 admin admin] PATH_INFO: `/v3/projects/681951f0d34c493081df7168f0bf06ba/users/05e9417caf3b48699f9686828b499e6c/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.287117 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3542ab1-d495-419e-96f7-426a1460b7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681951f0d34c493081df7168f0bf06ba, user_id=05e9417caf3b48699f9686828b499e6c, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.289752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3542ab1-d495-419e-96f7-426a1460b7f5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.309584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1344/2712] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:43 2022] PUT /identity/v3/projects/681951f0d34c493081df7168f0bf06ba/users/05e9417caf3b48699f9686828b499e6c/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:43.329088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710a7e8-1313-4c0f-800d-2c948b381f5f None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.329311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710a7e8-1313-4c0f-800d-2c948b381f5f None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.329592 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710a7e8-1313-4c0f-800d-2c948b381f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.359411 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a710a7e8-1313-4c0f-800d-2c948b381f5f None None] MFA Rules not processed for user `05e9417caf3b48699f9686828b499e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:43.360753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a710a7e8-1313-4c0f-800d-2c948b381f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:43.373313 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:43.429727 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1369/2713] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:43.434547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3542ab1-d495-419e-96f7-426a1460b7f5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:43.503284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3542ab1-d495-419e-96f7-426a1460b7f5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:43.509058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacde521-e3f7-4e30-8fea-6234627ab9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:43.509722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacde521-e3f7-4e30-8fea-6234627ab9e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.510002 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacde521-e3f7-4e30-8fea-6234627ab9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.510272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacde521-e3f7-4e30-8fea-6234627ab9e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.511344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacde521-e3f7-4e30-8fea-6234627ab9e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.513275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacde521-e3f7-4e30-8fea-6234627ab9e8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.542176 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1345/2714] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:43 2022] POST /identity/v3/projects => generated 395 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:43.546399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710a7e8-1313-4c0f-800d-2c948b381f5f None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:43.557889 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710a7e8-1313-4c0f-800d-2c948b381f5f None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:43.616194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39544a4-dc97-4325-87ac-d2de4f9fa812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:43.616871 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39544a4-dc97-4325-87ac-d2de4f9fa812 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.617178 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39544a4-dc97-4325-87ac-d2de4f9fa812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.617428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39544a4-dc97-4325-87ac-d2de4f9fa812 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.618135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39544a4-dc97-4325-87ac-d2de4f9fa812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.620210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39544a4-dc97-4325-87ac-d2de4f9fa812 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.647774 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1370/2715] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:43 2022] POST /identity/v3/users => generated 304 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:43.654055 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacde521-e3f7-4e30-8fea-6234627ab9e8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:43.666653 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacde521-e3f7-4e30-8fea-6234627ab9e8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:43.672468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5ee5d-b2ab-4826-9699-e219843a0dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:43.673344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5ee5d-b2ab-4826-9699-e219843a0dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.673630 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5ee5d-b2ab-4826-9699-e219843a0dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.673903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5ee5d-b2ab-4826-9699-e219843a0dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.674632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5ee5d-b2ab-4826-9699-e219843a0dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.676259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5ee5d-b2ab-4826-9699-e219843a0dbb admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.687214 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1346/2716] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:43.691564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39544a4-dc97-4325-87ac-d2de4f9fa812 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:43.702743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39544a4-dc97-4325-87ac-d2de4f9fa812 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:43.709594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883814d-fde5-4cb0-b25b-09b2920bcdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:43.710927 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883814d-fde5-4cb0-b25b-09b2920bcdb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.711294 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883814d-fde5-4cb0-b25b-09b2920bcdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.711595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883814d-fde5-4cb0-b25b-09b2920bcdb6 admin admin] PATH_INFO: `/v3/projects/076c77c94108400b9bca28320d78bd21/users/0352dc7d76d44045b224c6f1f30eed60/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.729956 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883814d-fde5-4cb0-b25b-09b2920bcdb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076c77c94108400b9bca28320d78bd21, user_id=0352dc7d76d44045b224c6f1f30eed60, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.732723 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883814d-fde5-4cb0-b25b-09b2920bcdb6 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.748892 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1371/2717] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:43 2022] PUT /identity/v3/projects/076c77c94108400b9bca28320d78bd21/users/0352dc7d76d44045b224c6f1f30eed60/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:43.753140 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5ee5d-b2ab-4826-9699-e219843a0dbb admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:43.763282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5ee5d-b2ab-4826-9699-e219843a0dbb admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:43.797825 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:43.826742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace2feb-2ce1-44ca-a2e9-64bebc6a28a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:43.827438 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace2feb-2ce1-44ca-a2e9-64bebc6a28a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.827725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace2feb-2ce1-44ca-a2e9-64bebc6a28a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.828005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace2feb-2ce1-44ca-a2e9-64bebc6a28a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.828749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace2feb-2ce1-44ca-a2e9-64bebc6a28a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.830566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace2feb-2ce1-44ca-a2e9-64bebc6a28a0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.841917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1347/2718] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:43.846325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883814d-fde5-4cb0-b25b-09b2920bcdb6 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:43.856906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883814d-fde5-4cb0-b25b-09b2920bcdb6 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:43.862106 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e05e1f-4826-4cdc-82cd-76fb25db9e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:43.863025 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e05e1f-4826-4cdc-82cd-76fb25db9e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.863305 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e05e1f-4826-4cdc-82cd-76fb25db9e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.863579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e05e1f-4826-4cdc-82cd-76fb25db9e56 admin admin] PATH_INFO: `/v3/projects/076c77c94108400b9bca28320d78bd21/users/0352dc7d76d44045b224c6f1f30eed60/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.865582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e05e1f-4826-4cdc-82cd-76fb25db9e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076c77c94108400b9bca28320d78bd21, user_id=0352dc7d76d44045b224c6f1f30eed60, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:43.868318 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e05e1f-4826-4cdc-82cd-76fb25db9e56 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:43.878633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1372/2719] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:43 2022] PUT /identity/v3/projects/076c77c94108400b9bca28320d78bd21/users/0352dc7d76d44045b224c6f1f30eed60/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:43.888984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c70df-4806-4f12-9a56-c37548d7a289 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:43.889288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c70df-4806-4f12-9a56-c37548d7a289 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:43.889571 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c70df-4806-4f12-9a56-c37548d7a289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:43.917376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cd7c70df-4806-4f12-9a56-c37548d7a289 None None] MFA Rules not processed for user `0352dc7d76d44045b224c6f1f30eed60`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:43.918616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cd7c70df-4806-4f12-9a56-c37548d7a289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:43.991909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1348/2720] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:44.003177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e05e1f-4826-4cdc-82cd-76fb25db9e56 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:44.005331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a5e05e1f-4826-4cdc-82cd-76fb25db9e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:44.066950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e05e1f-4826-4cdc-82cd-76fb25db9e56 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:44.071311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df222a4-cab9-4efe-a5b0-daa5892bdc02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:44.071959 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df222a4-cab9-4efe-a5b0-daa5892bdc02 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:44.072236 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df222a4-cab9-4efe-a5b0-daa5892bdc02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:44.072507 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df222a4-cab9-4efe-a5b0-daa5892bdc02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:44.081857 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df222a4-cab9-4efe-a5b0-daa5892bdc02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:44.083328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df222a4-cab9-4efe-a5b0-daa5892bdc02 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:44.093105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1373/2721] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:44.180598 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c70df-4806-4f12-9a56-c37548d7a289 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:44.191549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c70df-4806-4f12-9a56-c37548d7a289 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:44.195501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a16b3-8df8-40b4-b617-7530204097ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:44.196151 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a16b3-8df8-40b4-b617-7530204097ff service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:44.196431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a16b3-8df8-40b4-b617-7530204097ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:44.196702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a16b3-8df8-40b4-b617-7530204097ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:44.258232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a16b3-8df8-40b4-b617-7530204097ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:44.259691 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a16b3-8df8-40b4-b617-7530204097ff service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:44.268734 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1349/2722] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:47.928088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df222a4-cab9-4efe-a5b0-daa5892bdc02 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:47.930569 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8df222a4-cab9-4efe-a5b0-daa5892bdc02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:47.947345 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:47.997493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df222a4-cab9-4efe-a5b0-daa5892bdc02 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:48.002040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3d46d-1a9c-43d5-8de3-3129ace2a8c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:48.002717 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3d46d-1a9c-43d5-8de3-3129ace2a8c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:48.003000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3d46d-1a9c-43d5-8de3-3129ace2a8c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:48.003271 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3d46d-1a9c-43d5-8de3-3129ace2a8c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:48.005390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-65f3d46d-1a9c-43d5-8de3-3129ace2a8c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:48.093866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3d46d-1a9c-43d5-8de3-3129ace2a8c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:48.095387 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3d46d-1a9c-43d5-8de3-3129ace2a8c0 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:48.114051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1374/2723] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:48.412634 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a16b3-8df8-40b4-b617-7530204097ff service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:48.467005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=100 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:48.478870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a16b3-8df8-40b4-b617-7530204097ff service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:48.483512 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:48.484533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:48.484856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:48.485132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] PATH_INFO: `/v3/users/05e9417caf3b48699f9686828b499e6c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:48.486336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e9417caf3b48699f9686828b499e6c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:48.488365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:48.539329 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e9417caf3b48699f9686828b499e6c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:48.540075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e9417caf3b48699f9686828b499e6c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:48.554075 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e9417caf3b48699f9686828b499e6c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:48.554875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e9417caf3b48699f9686828b499e6c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:48.564469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e9417caf3b48699f9686828b499e6c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:48.579655 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1350/2724] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:48 2022] DELETE /identity/v3/users/05e9417caf3b48699f9686828b499e6c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:48.584304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c040a1-055c-41a9-8abc-ee153a8a90e8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:48.584594 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c040a1-055c-41a9-8abc-ee153a8a90e8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:48.585210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c040a1-055c-41a9-8abc-ee153a8a90e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:48.602973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-12c040a1-055c-41a9-8abc-ee153a8a90e8 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:48.604179 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-12c040a1-055c-41a9-8abc-ee153a8a90e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:48.664245 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1375/2725] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:48.674350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:48.675960 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:48.737648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd56bd-7a79-480e-8307-85974af1daa0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:48.742159 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535513c9-ccbe-400a-b545-da1ce7d1101b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:48.742857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535513c9-ccbe-400a-b545-da1ce7d1101b service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:48.743145 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535513c9-ccbe-400a-b545-da1ce7d1101b service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:48.743405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535513c9-ccbe-400a-b545-da1ce7d1101b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:48.753582 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535513c9-ccbe-400a-b545-da1ce7d1101b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:48.755763 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535513c9-ccbe-400a-b545-da1ce7d1101b service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:48.765873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1351/2726] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:49.051622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c040a1-055c-41a9-8abc-ee153a8a90e8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:49.055574 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-12c040a1-055c-41a9-8abc-ee153a8a90e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:49.069699 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c040a1-055c-41a9-8abc-ee153a8a90e8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:49.081175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:49.083348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:49.083628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:49.084079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] PATH_INFO: `/v3/projects/681951f0d34c493081df7168f0bf06ba` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:49.085573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=681951f0d34c493081df7168f0bf06ba)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:49.089217 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:49.136509 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:49.162961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681951f0d34c493081df7168f0bf06ba', 'request_id': 'req-12bae93d-e0a1-40af-a8b3-81e629141e50'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.170648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681951f0d34c493081df7168f0bf06ba', 'request_id': 'req-12bae93d-e0a1-40af-a8b3-81e629141e50'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.177917 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681951f0d34c493081df7168f0bf06ba', 'request_id': 'req-12bae93d-e0a1-40af-a8b3-81e629141e50'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.198669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] The token cache is being invalidate because project 681951f0d34c493081df7168f0bf06ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:49.199327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.202384 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.207702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1376/2727] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:49 2022] DELETE /identity/v3/projects/681951f0d34c493081df7168f0bf06ba => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:49.209592 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535513c9-ccbe-400a-b545-da1ce7d1101b service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:49.224377 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535513c9-ccbe-400a-b545-da1ce7d1101b service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:49.298575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88763b4-10d4-49ab-9ce8-c47fa5c2c765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:49.299489 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88763b4-10d4-49ab-9ce8-c47fa5c2c765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:49.299831 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88763b4-10d4-49ab-9ce8-c47fa5c2c765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:49.300168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88763b4-10d4-49ab-9ce8-c47fa5c2c765 admin admin] PATH_INFO: `/v3/users/0352dc7d76d44045b224c6f1f30eed60` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:49.301709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88763b4-10d4-49ab-9ce8-c47fa5c2c765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0352dc7d76d44045b224c6f1f30eed60)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:49.303792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88763b4-10d4-49ab-9ce8-c47fa5c2c765 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:49.327153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:49.375030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e88763b4-10d4-49ab-9ce8-c47fa5c2c765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0352dc7d76d44045b224c6f1f30eed60'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.375814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e88763b4-10d4-49ab-9ce8-c47fa5c2c765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0352dc7d76d44045b224c6f1f30eed60'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.391016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e88763b4-10d4-49ab-9ce8-c47fa5c2c765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0352dc7d76d44045b224c6f1f30eed60'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.391774 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e88763b4-10d4-49ab-9ce8-c47fa5c2c765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0352dc7d76d44045b224c6f1f30eed60'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.402670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-e88763b4-10d4-49ab-9ce8-c47fa5c2c765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0352dc7d76d44045b224c6f1f30eed60'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.418193 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1352/2728] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:49 2022] DELETE /identity/v3/users/0352dc7d76d44045b224c6f1f30eed60 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:49.734521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:49.736053 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:49.749820 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bae93d-e0a1-40af-a8b3-81e629141e50 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:49.814005 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:49.815204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:49.815483 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:49.815761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] PATH_INFO: `/v3/projects/076c77c94108400b9bca28320d78bd21` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:49.817070 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=076c77c94108400b9bca28320d78bd21)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:49.819227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:49.868328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076c77c94108400b9bca28320d78bd21', 'request_id': 'req-49767a2c-3243-4392-b793-8cc2ffe7c018'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.874624 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076c77c94108400b9bca28320d78bd21', 'request_id': 'req-49767a2c-3243-4392-b793-8cc2ffe7c018'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.881532 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076c77c94108400b9bca28320d78bd21', 'request_id': 'req-49767a2c-3243-4392-b793-8cc2ffe7c018'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.894363 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] The token cache is being invalidate because project 076c77c94108400b9bca28320d78bd21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:49.894644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.895462 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-49767a2c-3243-4392-b793-8cc2ffe7c018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:49.897100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1377/2729] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:49 2022] DELETE /identity/v3/projects/076c77c94108400b9bca28320d78bd21 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:49.901997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2f65d-ec1b-41bb-a472-b3f5b8cbc12c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:49.902289 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2f65d-ec1b-41bb-a472-b3f5b8cbc12c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:49.902570 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2f65d-ec1b-41bb-a472-b3f5b8cbc12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:49.924050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-95f2f65d-ec1b-41bb-a472-b3f5b8cbc12c None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:49.925295 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-95f2f65d-ec1b-41bb-a472-b3f5b8cbc12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:49.991326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1353/2730] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:50.000247 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db768af7-3aef-4570-82bd-9cf22a4af7cf None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.000683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db768af7-3aef-4570-82bd-9cf22a4af7cf None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.000998 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db768af7-3aef-4570-82bd-9cf22a4af7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.020978 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-db768af7-3aef-4570-82bd-9cf22a4af7cf None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:50.022204 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-db768af7-3aef-4570-82bd-9cf22a4af7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:50.030199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1378/2731] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:50.034493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2f65d-ec1b-41bb-a472-b3f5b8cbc12c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:50.044589 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2f65d-ec1b-41bb-a472-b3f5b8cbc12c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:50.049271 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6325470-6015-4125-b350-5535f608559d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:50.050011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6325470-6015-4125-b350-5535f608559d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.050360 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6325470-6015-4125-b350-5535f608559d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.050718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6325470-6015-4125-b350-5535f608559d admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.051855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6325470-6015-4125-b350-5535f608559d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:50.052253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6325470-6015-4125-b350-5535f608559d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:50.053929 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6325470-6015-4125-b350-5535f608559d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:50.066186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1354/2732] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:49:50.070818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db768af7-3aef-4570-82bd-9cf22a4af7cf None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:50.083684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db768af7-3aef-4570-82bd-9cf22a4af7cf None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:50.089527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2314c9-3c2c-46e8-8b15-8615aa7e31bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:50.090199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2314c9-3c2c-46e8-8b15-8615aa7e31bd admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.090480 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2314c9-3c2c-46e8-8b15-8615aa7e31bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.090783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2314c9-3c2c-46e8-8b15-8615aa7e31bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.091493 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2314c9-3c2c-46e8-8b15-8615aa7e31bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:50.093419 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2314c9-3c2c-46e8-8b15-8615aa7e31bd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:50.122654 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1379/2733] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:50 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:50.126953 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6325470-6015-4125-b350-5535f608559d admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:50.137044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6325470-6015-4125-b350-5535f608559d admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:50.192450 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22dd79-7839-4087-99be-2d39cb8c9038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:50.193098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22dd79-7839-4087-99be-2d39cb8c9038 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.193386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22dd79-7839-4087-99be-2d39cb8c9038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.193657 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22dd79-7839-4087-99be-2d39cb8c9038 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.194350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22dd79-7839-4087-99be-2d39cb8c9038 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:50.196322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae22dd79-7839-4087-99be-2d39cb8c9038 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:50.221253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1355/2734] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:50 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:50.225748 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2314c9-3c2c-46e8-8b15-8615aa7e31bd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:50.236066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2314c9-3c2c-46e8-8b15-8615aa7e31bd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:50.241404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388ea3d-539f-42ea-9876-e236a1450b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:50.242069 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ea3d-539f-42ea-9876-e236a1450b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.242347 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ea3d-539f-42ea-9876-e236a1450b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.242775 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388ea3d-539f-42ea-9876-e236a1450b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.243510 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388ea3d-539f-42ea-9876-e236a1450b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:50.245177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2388ea3d-539f-42ea-9876-e236a1450b90 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:50.256621 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1380/2735] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:50.260949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22dd79-7839-4087-99be-2d39cb8c9038 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:50.272380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae22dd79-7839-4087-99be-2d39cb8c9038 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:50.278340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8aa82f-f35c-4209-83f5-154440e99528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:50.279292 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8aa82f-f35c-4209-83f5-154440e99528 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.279735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8aa82f-f35c-4209-83f5-154440e99528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.280044 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8aa82f-f35c-4209-83f5-154440e99528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.281288 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8aa82f-f35c-4209-83f5-154440e99528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:50.283293 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8aa82f-f35c-4209-83f5-154440e99528 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:50.294004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:50.303079 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1356/2736] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:50.305327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388ea3d-539f-42ea-9876-e236a1450b90 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:50.317558 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388ea3d-539f-42ea-9876-e236a1450b90 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:50.322437 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8aeb6b-6d41-499d-8ef8-8547827ea620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:50.323388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8aeb6b-6d41-499d-8ef8-8547827ea620 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.323671 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8aeb6b-6d41-499d-8ef8-8547827ea620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.323951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8aeb6b-6d41-499d-8ef8-8547827ea620 admin admin] PATH_INFO: `/v3/projects/95588afdcc654a1faf11d9f1d91882e2/users/321319cc654f4b32b77f1babf69250f3/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.334138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:50.343020 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8aeb6b-6d41-499d-8ef8-8547827ea620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95588afdcc654a1faf11d9f1d91882e2, user_id=321319cc654f4b32b77f1babf69250f3, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:50.346083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8aeb6b-6d41-499d-8ef8-8547827ea620 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:50.361550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1381/2737] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:50 2022] PUT /identity/v3/projects/95588afdcc654a1faf11d9f1d91882e2/users/321319cc654f4b32b77f1babf69250f3/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:50.372046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63584153-6c4e-4acf-9826-823aa8e94af2 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.372330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63584153-6c4e-4acf-9826-823aa8e94af2 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.372611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63584153-6c4e-4acf-9826-823aa8e94af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.399114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-63584153-6c4e-4acf-9826-823aa8e94af2 None None] MFA Rules not processed for user `321319cc654f4b32b77f1babf69250f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:50.400323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-63584153-6c4e-4acf-9826-823aa8e94af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:50.472039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1357/2738] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:50.483847 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8aeb6b-6d41-499d-8ef8-8547827ea620 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:50.485840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7a8aeb6b-6d41-499d-8ef8-8547827ea620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:50.544434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8aeb6b-6d41-499d-8ef8-8547827ea620 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:50.549242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86afb50-5abf-445b-923d-e354630794f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:50.549972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86afb50-5abf-445b-923d-e354630794f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.550320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86afb50-5abf-445b-923d-e354630794f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.550659 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86afb50-5abf-445b-923d-e354630794f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.560676 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86afb50-5abf-445b-923d-e354630794f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:50.562117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86afb50-5abf-445b-923d-e354630794f6 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:50.572722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1382/2739] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:50.597603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63584153-6c4e-4acf-9826-823aa8e94af2 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:50.666161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63584153-6c4e-4acf-9826-823aa8e94af2 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:50.671256 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:50.672161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.672447 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.672719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] PATH_INFO: `/v3/users/321319cc654f4b32b77f1babf69250f3` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.674027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=321319cc654f4b32b77f1babf69250f3)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:50.678215 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:50.732736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '321319cc654f4b32b77f1babf69250f3'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:50.733396 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '321319cc654f4b32b77f1babf69250f3'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:50.746720 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '321319cc654f4b32b77f1babf69250f3'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:50.747454 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '321319cc654f4b32b77f1babf69250f3'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:50.756670 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '321319cc654f4b32b77f1babf69250f3'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:50.770018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1358/2740] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:50 2022] DELETE /identity/v3/users/321319cc654f4b32b77f1babf69250f3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:50.775333 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4817b-6cee-4f71-8f14-d6715f870cb8 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.775702 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4817b-6cee-4f71-8f14-d6715f870cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.776441 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4817b-6cee-4f71-8f14-d6715f870cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.811406 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-a0b4817b-6cee-4f71-8f14-d6715f870cb8 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:50.812665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a0b4817b-6cee-4f71-8f14-d6715f870cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:50.879158 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1383/2741] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 08:49:50.889465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:50.890951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:50.902563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:50.955978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdea925-3fc1-4599-bdba-0e474b0b7d35 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:50.960580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aeb9e2-2e73-4f99-b0c9-10e124edeb08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:50.961232 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aeb9e2-2e73-4f99-b0c9-10e124edeb08 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:50.961501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aeb9e2-2e73-4f99-b0c9-10e124edeb08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:50.961775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aeb9e2-2e73-4f99-b0c9-10e124edeb08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:50.972327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aeb9e2-2e73-4f99-b0c9-10e124edeb08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:50.973811 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aeb9e2-2e73-4f99-b0c9-10e124edeb08 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:50.983578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1359/2742] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:51.276428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4817b-6cee-4f71-8f14-d6715f870cb8 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:51.278151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a0b4817b-6cee-4f71-8f14-d6715f870cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:51.292459 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4817b-6cee-4f71-8f14-d6715f870cb8 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:51.298274 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:51.299328 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:51.299743 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:51.300140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] PATH_INFO: `/v3/projects/95588afdcc654a1faf11d9f1d91882e2` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:51.301942 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95588afdcc654a1faf11d9f1d91882e2)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:51.304220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:51.360449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95588afdcc654a1faf11d9f1d91882e2', 'request_id': 'req-14143b51-2fa7-44a8-a2ba-fc464304a8c4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:51.367514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95588afdcc654a1faf11d9f1d91882e2', 'request_id': 'req-14143b51-2fa7-44a8-a2ba-fc464304a8c4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:51.373251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95588afdcc654a1faf11d9f1d91882e2', 'request_id': 'req-14143b51-2fa7-44a8-a2ba-fc464304a8c4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:51.387390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] The token cache is being invalidate because project 95588afdcc654a1faf11d9f1d91882e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:49:51.387688 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:51.388371 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-14143b51-2fa7-44a8-a2ba-fc464304a8c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:51.389938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1384/2743] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:49:51 2022] DELETE /identity/v3/projects/95588afdcc654a1faf11d9f1d91882e2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:51.395293 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a48e-f204-4d75-81ae-02937c6db8d7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:51.395583 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a48e-f204-4d75-81ae-02937c6db8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:51.395881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a48e-f204-4d75-81ae-02937c6db8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:51.414501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-cee1a48e-f204-4d75-81ae-02937c6db8d7 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:51.415841 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-cee1a48e-f204-4d75-81ae-02937c6db8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:51.486996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1360/2744] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:51.496006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115cd07-dcf7-44e3-9df8-3aa60a4e9d84 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:51.496656 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115cd07-dcf7-44e3-9df8-3aa60a4e9d84 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:51.496991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115cd07-dcf7-44e3-9df8-3aa60a4e9d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:51.516929 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-0115cd07-dcf7-44e3-9df8-3aa60a4e9d84 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:51.518298 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-0115cd07-dcf7-44e3-9df8-3aa60a4e9d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:51.526995 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1385/2745] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:51.531540 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1a48e-f204-4d75-81ae-02937c6db8d7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:51.542307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1a48e-f204-4d75-81ae-02937c6db8d7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:51.547839 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9923e13-4e10-45b3-9229-feffb10fdfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:51.548859 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9923e13-4e10-45b3-9229-feffb10fdfda admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:51.549019 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9923e13-4e10-45b3-9229-feffb10fdfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:51.549331 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9923e13-4e10-45b3-9229-feffb10fdfda admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:51.550168 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9923e13-4e10-45b3-9229-feffb10fdfda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:49:51.550486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9923e13-4e10-45b3-9229-feffb10fdfda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:51.552550 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9923e13-4e10-45b3-9229-feffb10fdfda admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:51.565614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1361/2746] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:49:51.570452 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115cd07-dcf7-44e3-9df8-3aa60a4e9d84 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:51.577722 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:51.582635 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115cd07-dcf7-44e3-9df8-3aa60a4e9d84 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:51.589776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a43d9-2915-4d34-8f2e-04492c030e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:51.590461 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a43d9-2915-4d34-8f2e-04492c030e06 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:51.590753 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a43d9-2915-4d34-8f2e-04492c030e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:51.591046 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a43d9-2915-4d34-8f2e-04492c030e06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:51.592050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a43d9-2915-4d34-8f2e-04492c030e06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:51.594469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a43d9-2915-4d34-8f2e-04492c030e06 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:51.625099 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1386/2747] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:51 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:51.629741 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9923e13-4e10-45b3-9229-feffb10fdfda admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:51.641064 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9923e13-4e10-45b3-9229-feffb10fdfda admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:51.704818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f464a-c9c1-42d5-b66b-e257c2d30ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:51.704818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f464a-c9c1-42d5-b66b-e257c2d30ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:51.704818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f464a-c9c1-42d5-b66b-e257c2d30ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:51.704818 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f464a-c9c1-42d5-b66b-e257c2d30ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:51.705387 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f464a-c9c1-42d5-b66b-e257c2d30ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:51.707029 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f464a-c9c1-42d5-b66b-e257c2d30ff0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:51.731171 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1362/2748] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:51 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:51.735465 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a43d9-2915-4d34-8f2e-04492c030e06 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:51.746296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a43d9-2915-4d34-8f2e-04492c030e06 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:51.751067 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ba9ae-03c4-4ddd-b918-2bc38b173938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:51.751754 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ba9ae-03c4-4ddd-b918-2bc38b173938 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:51.752023 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ba9ae-03c4-4ddd-b918-2bc38b173938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:51.752336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768ba9ae-03c4-4ddd-b918-2bc38b173938 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:51.753074 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ba9ae-03c4-4ddd-b918-2bc38b173938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:51.754769 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768ba9ae-03c4-4ddd-b918-2bc38b173938 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:51.765696 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1387/2749] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:51.770013 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f464a-c9c1-42d5-b66b-e257c2d30ff0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:51.781264 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f464a-c9c1-42d5-b66b-e257c2d30ff0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:51.786846 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78971153-7613-41e9-acc4-0364d8e15ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:51.787596 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78971153-7613-41e9-acc4-0364d8e15ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:51.787936 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78971153-7613-41e9-acc4-0364d8e15ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:51.788317 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78971153-7613-41e9-acc4-0364d8e15ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:51.789128 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78971153-7613-41e9-acc4-0364d8e15ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:51.790820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78971153-7613-41e9-acc4-0364d8e15ce6 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:51.801616 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1363/2750] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:51.805922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ba9ae-03c4-4ddd-b918-2bc38b173938 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:51.819075 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768ba9ae-03c4-4ddd-b918-2bc38b173938 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:51.825141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934f93a-4955-4ec9-8666-0de1798b8e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:51.826044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934f93a-4955-4ec9-8666-0de1798b8e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:51.826324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934f93a-4955-4ec9-8666-0de1798b8e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:51.826651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934f93a-4955-4ec9-8666-0de1798b8e05 admin admin] PATH_INFO: `/v3/projects/659c0842a1be47248bb57eb2fdd13d3d/users/bb4862e82476431f80764f410fa18fc7/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:51.843867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934f93a-4955-4ec9-8666-0de1798b8e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=659c0842a1be47248bb57eb2fdd13d3d, user_id=bb4862e82476431f80764f410fa18fc7, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:51.846550 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934f93a-4955-4ec9-8666-0de1798b8e05 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:51.871835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1388/2751] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:51 2022] PUT /identity/v3/projects/659c0842a1be47248bb57eb2fdd13d3d/users/bb4862e82476431f80764f410fa18fc7/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:51.873815 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d3e90-ded0-4a78-941b-bd061930281e None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:51.874179 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d3e90-ded0-4a78-941b-bd061930281e None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:51.874894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d3e90-ded0-4a78-941b-bd061930281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:51.907687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-ac0d3e90-ded0-4a78-941b-bd061930281e None None] MFA Rules not processed for user `bb4862e82476431f80764f410fa18fc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:51.908843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ac0d3e90-ded0-4a78-941b-bd061930281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:51.916951 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:51.990294 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1364/2752] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:51.994900 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934f93a-4955-4ec9-8666-0de1798b8e05 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:52.073789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934f93a-4955-4ec9-8666-0de1798b8e05 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:52.078994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2675507-c1dc-4806-9d32-8d56d4e97b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:52.079393 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2675507-c1dc-4806-9d32-8d56d4e97b3e admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.079725 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2675507-c1dc-4806-9d32-8d56d4e97b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.080167 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2675507-c1dc-4806-9d32-8d56d4e97b3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.080736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2675507-c1dc-4806-9d32-8d56d4e97b3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:52.084085 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2675507-c1dc-4806-9d32-8d56d4e97b3e admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:52.114866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1389/2753] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:51 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:49:52.116857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d3e90-ded0-4a78-941b-bd061930281e None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:52.128283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d3e90-ded0-4a78-941b-bd061930281e None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:52.186131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf8e26-9c0e-4cc7-9999-d62b765d4607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:52.186819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf8e26-9c0e-4cc7-9999-d62b765d4607 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.187100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf8e26-9c0e-4cc7-9999-d62b765d4607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.187373 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf8e26-9c0e-4cc7-9999-d62b765d4607 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.188077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf8e26-9c0e-4cc7-9999-d62b765d4607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:52.189978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf8e26-9c0e-4cc7-9999-d62b765d4607 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:52.211412 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1365/2754] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:52 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:52.216309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2675507-c1dc-4806-9d32-8d56d4e97b3e admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:52.231920 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2675507-c1dc-4806-9d32-8d56d4e97b3e admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:52.239166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee87a8-631d-47e2-882f-5f57d9e999bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:52.239830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee87a8-631d-47e2-882f-5f57d9e999bc admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.240114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee87a8-631d-47e2-882f-5f57d9e999bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.240391 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee87a8-631d-47e2-882f-5f57d9e999bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.241156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee87a8-631d-47e2-882f-5f57d9e999bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:52.242865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee87a8-631d-47e2-882f-5f57d9e999bc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:52.254283 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1390/2755] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:52.260338 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf8e26-9c0e-4cc7-9999-d62b765d4607 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:52.277770 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf8e26-9c0e-4cc7-9999-d62b765d4607 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:52.284185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126273f5-4541-4b81-8ed1-041ac0bab87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:52.285142 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126273f5-4541-4b81-8ed1-041ac0bab87d admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.285597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126273f5-4541-4b81-8ed1-041ac0bab87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.285993 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126273f5-4541-4b81-8ed1-041ac0bab87d admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.287144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126273f5-4541-4b81-8ed1-041ac0bab87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:52.289749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126273f5-4541-4b81-8ed1-041ac0bab87d admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:52.305350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1366/2756] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:52.310114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee87a8-631d-47e2-882f-5f57d9e999bc admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:52.321270 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee87a8-631d-47e2-882f-5f57d9e999bc admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:52.327049 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f756e-3bde-4537-bbeb-64bf7b82b9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:52.327923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f756e-3bde-4537-bbeb-64bf7b82b9e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.328203 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f756e-3bde-4537-bbeb-64bf7b82b9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.328476 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f756e-3bde-4537-bbeb-64bf7b82b9e8 admin admin] PATH_INFO: `/v3/projects/615f19017a4f4de999639cc12d36ed0d/users/af26d817433a44b9ada4a940d3cb25fd/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.349485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f756e-3bde-4537-bbeb-64bf7b82b9e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615f19017a4f4de999639cc12d36ed0d, user_id=af26d817433a44b9ada4a940d3cb25fd, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:52.352249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f756e-3bde-4537-bbeb-64bf7b82b9e8 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:52.370279 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1391/2757] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:52 2022] PUT /identity/v3/projects/615f19017a4f4de999639cc12d36ed0d/users/af26d817433a44b9ada4a940d3cb25fd/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:52.382008 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67024149-b2c9-4b6a-81a4-ba2635c34418 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.382419 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67024149-b2c9-4b6a-81a4-ba2635c34418 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.382732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67024149-b2c9-4b6a-81a4-ba2635c34418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.412986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-67024149-b2c9-4b6a-81a4-ba2635c34418 None None] MFA Rules not processed for user `af26d817433a44b9ada4a940d3cb25fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:52.414313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-67024149-b2c9-4b6a-81a4-ba2635c34418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:52.498849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1367/2758] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:52.501693 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f756e-3bde-4537-bbeb-64bf7b82b9e8 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:52.569187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:52.573186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f756e-3bde-4537-bbeb-64bf7b82b9e8 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:52.578939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdff89-56ed-490e-8351-6dd327a14f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:52.579611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdff89-56ed-490e-8351-6dd327a14f04 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.579893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdff89-56ed-490e-8351-6dd327a14f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.580165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdff89-56ed-490e-8351-6dd327a14f04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.580887 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdff89-56ed-490e-8351-6dd327a14f04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:52.583210 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdff89-56ed-490e-8351-6dd327a14f04 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:52.611884 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1392/2759] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:49:52 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:52.616543 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67024149-b2c9-4b6a-81a4-ba2635c34418 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:52.628306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67024149-b2c9-4b6a-81a4-ba2635c34418 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:52.640932 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:52.687654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2dec1-e2e0-43e1-a065-c460705f5c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:52.688333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2dec1-e2e0-43e1-a065-c460705f5c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.688618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2dec1-e2e0-43e1-a065-c460705f5c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.688892 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2dec1-e2e0-43e1-a065-c460705f5c2f admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.689597 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2dec1-e2e0-43e1-a065-c460705f5c2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:52.691577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2dec1-e2e0-43e1-a065-c460705f5c2f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:52.712888 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1368/2760] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:49:52 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:49:52.717435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdff89-56ed-490e-8351-6dd327a14f04 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:52.728703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdff89-56ed-490e-8351-6dd327a14f04 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:52.733682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d43c3-d04c-46dd-8283-b2c558890c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:52.734348 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d43c3-d04c-46dd-8283-b2c558890c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.734636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d43c3-d04c-46dd-8283-b2c558890c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.734941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d43c3-d04c-46dd-8283-b2c558890c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.735670 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d43c3-d04c-46dd-8283-b2c558890c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:52.737281 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d43c3-d04c-46dd-8283-b2c558890c0f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:52.748198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1393/2761] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:52.752684 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2dec1-e2e0-43e1-a065-c460705f5c2f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:52.763715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2dec1-e2e0-43e1-a065-c460705f5c2f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:52.769333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf0b1b-bb1b-46c2-b10f-bb75c9422b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:52.770809 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf0b1b-bb1b-46c2-b10f-bb75c9422b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.771098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf0b1b-bb1b-46c2-b10f-bb75c9422b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.771382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf0b1b-bb1b-46c2-b10f-bb75c9422b91 admin admin] PATH_INFO: `/v3/projects/0934449a239b482b916556d63430ed33/users/9aeeb8ddf1a149968ccac8e679080d46/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.787996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf0b1b-bb1b-46c2-b10f-bb75c9422b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0934449a239b482b916556d63430ed33, user_id=9aeeb8ddf1a149968ccac8e679080d46, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:52.791006 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf0b1b-bb1b-46c2-b10f-bb75c9422b91 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:52.808901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1369/2762] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:52 2022] PUT /identity/v3/projects/0934449a239b482b916556d63430ed33/users/9aeeb8ddf1a149968ccac8e679080d46/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:52.812894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d43c3-d04c-46dd-8283-b2c558890c0f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:52.823787 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d43c3-d04c-46dd-8283-b2c558890c0f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:52.886446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3969b3-a788-41e7-b718-99353aaa41ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:52.887661 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3969b3-a788-41e7-b718-99353aaa41ce admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.888014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3969b3-a788-41e7-b718-99353aaa41ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.888359 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3969b3-a788-41e7-b718-99353aaa41ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.889170 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3969b3-a788-41e7-b718-99353aaa41ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:52.890888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3969b3-a788-41e7-b718-99353aaa41ce admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:52.902731 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1394/2763] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:49:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:49:52.906987 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf0b1b-bb1b-46c2-b10f-bb75c9422b91 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:52.920622 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf0b1b-bb1b-46c2-b10f-bb75c9422b91 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:52.925501 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc7acf-0f8f-45cd-8287-7c043a5a11f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:52.926654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc7acf-0f8f-45cd-8287-7c043a5a11f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.926830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc7acf-0f8f-45cd-8287-7c043a5a11f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.927094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc7acf-0f8f-45cd-8287-7c043a5a11f8 admin admin] PATH_INFO: `/v3/projects/0934449a239b482b916556d63430ed33/users/9aeeb8ddf1a149968ccac8e679080d46/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.929626 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc7acf-0f8f-45cd-8287-7c043a5a11f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0934449a239b482b916556d63430ed33, user_id=9aeeb8ddf1a149968ccac8e679080d46, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:52.932322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc7acf-0f8f-45cd-8287-7c043a5a11f8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:52.943186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1370/2764] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:49:52 2022] PUT /identity/v3/projects/0934449a239b482b916556d63430ed33/users/9aeeb8ddf1a149968ccac8e679080d46/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:52.953770 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcad606-2a65-4d0e-b0a4-bb2349bf8bfd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:52.954326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcad606-2a65-4d0e-b0a4-bb2349bf8bfd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:52.954610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcad606-2a65-4d0e-b0a4-bb2349bf8bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:52.984358 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7fcad606-2a65-4d0e-b0a4-bb2349bf8bfd None None] MFA Rules not processed for user `9aeeb8ddf1a149968ccac8e679080d46`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:52.985565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7fcad606-2a65-4d0e-b0a4-bb2349bf8bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:53.057168 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1395/2765] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:53.067776 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc7acf-0f8f-45cd-8287-7c043a5a11f8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:53.069562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8edc7acf-0f8f-45cd-8287-7c043a5a11f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:53.138005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc7acf-0f8f-45cd-8287-7c043a5a11f8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:53.142888 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45f550-e592-44df-81eb-82722e72ad9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:53.143536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45f550-e592-44df-81eb-82722e72ad9b service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:53.143822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45f550-e592-44df-81eb-82722e72ad9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:53.144100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45f550-e592-44df-81eb-82722e72ad9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:53.226132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45f550-e592-44df-81eb-82722e72ad9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:53.227766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45f550-e592-44df-81eb-82722e72ad9b service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:53.237345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1371/2766] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:54.753270 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcad606-2a65-4d0e-b0a4-bb2349bf8bfd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:54.764446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcad606-2a65-4d0e-b0a4-bb2349bf8bfd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:54.768744 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cedca-0261-4006-bff8-3d5b550c24d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:54.769398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cedca-0261-4006-bff8-3d5b550c24d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:54.769700 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cedca-0261-4006-bff8-3d5b550c24d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:54.769957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cedca-0261-4006-bff8-3d5b550c24d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:54.821187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:54.831430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cedca-0261-4006-bff8-3d5b550c24d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:54.833818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cedca-0261-4006-bff8-3d5b550c24d2 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:54.848786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1396/2767] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:55.417206 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45f550-e592-44df-81eb-82722e72ad9b service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:55.418756 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9d45f550-e592-44df-81eb-82722e72ad9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:55.431016 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45f550-e592-44df-81eb-82722e72ad9b service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:55.435286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaace928-c8dd-4a3a-9b17-458e8a401084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:55.435948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaace928-c8dd-4a3a-9b17-458e8a401084 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:55.436236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaace928-c8dd-4a3a-9b17-458e8a401084 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:55.436520 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaace928-c8dd-4a3a-9b17-458e8a401084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:55.446964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaace928-c8dd-4a3a-9b17-458e8a401084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:55.448405 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaace928-c8dd-4a3a-9b17-458e8a401084 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:55.459760 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1372/2768] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:49:59.386800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cedca-0261-4006-bff8-3d5b550c24d2 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:59.453573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cedca-0261-4006-bff8-3d5b550c24d2 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:59.458488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:59.459404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:59.459681 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:59.460057 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] PATH_INFO: `/v3/users/bb4862e82476431f80764f410fa18fc7` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:59.461212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb4862e82476431f80764f410fa18fc7)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:59.463213 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:59.522497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb4862e82476431f80764f410fa18fc7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:59.523197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb4862e82476431f80764f410fa18fc7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:59.535395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb4862e82476431f80764f410fa18fc7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:59.536030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb4862e82476431f80764f410fa18fc7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:59.544613 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb4862e82476431f80764f410fa18fc7'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:49:59.557760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1397/2769] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:49:59 2022] DELETE /identity/v3/users/bb4862e82476431f80764f410fa18fc7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:49:59.562432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec962e9-ee36-492e-8303-8058bf08b352 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:59.562952 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec962e9-ee36-492e-8303-8058bf08b352 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:59.563133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec962e9-ee36-492e-8303-8058bf08b352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:59.581820 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8ec962e9-ee36-492e-8303-8058bf08b352 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:49:59.583357 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8ec962e9-ee36-492e-8303-8058bf08b352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:59.587852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:49:59.646722 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1373/2770] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:49:59.657121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:49:59.658573 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:49:59.721895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b53d4a-a63b-4663-940e-6e75b3f59159 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:49:59.725758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2d06f-cafb-4f28-b81f-4eb0ddd8986d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:49:59.726400 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2d06f-cafb-4f28-b81f-4eb0ddd8986d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:49:59.726680 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2d06f-cafb-4f28-b81f-4eb0ddd8986d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:49:59.726988 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2d06f-cafb-4f28-b81f-4eb0ddd8986d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:49:59.736757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2d06f-cafb-4f28-b81f-4eb0ddd8986d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:49:59.738190 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2d06f-cafb-4f28-b81f-4eb0ddd8986d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:49:59.748042 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1398/2771] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:50:00.036971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec962e9-ee36-492e-8303-8058bf08b352 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:50:00.038467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8ec962e9-ee36-492e-8303-8058bf08b352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:00.051355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec962e9-ee36-492e-8303-8058bf08b352 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:50:00.057132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:50:00.058027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:00.058352 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:00.058627 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] PATH_INFO: `/v3/projects/659c0842a1be47248bb57eb2fdd13d3d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:00.060050 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] RBAC: Authorizing `identity:delete_project(project_id=659c0842a1be47248bb57eb2fdd13d3d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:50:00.062059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:50:00.112633 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659c0842a1be47248bb57eb2fdd13d3d', 'request_id': 'req-a3acef30-28a0-4b02-85ac-005d73f73bed'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.119380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659c0842a1be47248bb57eb2fdd13d3d', 'request_id': 'req-a3acef30-28a0-4b02-85ac-005d73f73bed'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.125239 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659c0842a1be47248bb57eb2fdd13d3d', 'request_id': 'req-a3acef30-28a0-4b02-85ac-005d73f73bed'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.138718 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] The token cache is being invalidate because project 659c0842a1be47248bb57eb2fdd13d3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:50:00.139018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.139753 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.141462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1374/2772] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:50:00 2022] DELETE /identity/v3/projects/659c0842a1be47248bb57eb2fdd13d3d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:50:00.145655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2d06f-cafb-4f28-b81f-4eb0ddd8986d service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:50:00.155880 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2d06f-cafb-4f28-b81f-4eb0ddd8986d service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:50:00.165648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:50:00.220449 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:50:00.221326 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:00.221614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:00.221898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] PATH_INFO: `/v3/users/af26d817433a44b9ada4a940d3cb25fd` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:00.223267 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] RBAC: Authorizing `identity:delete_user(user_id=af26d817433a44b9ada4a940d3cb25fd)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:50:00.225250 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:50:00.274793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af26d817433a44b9ada4a940d3cb25fd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.275522 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af26d817433a44b9ada4a940d3cb25fd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.287630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af26d817433a44b9ada4a940d3cb25fd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.288310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af26d817433a44b9ada4a940d3cb25fd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.298242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af26d817433a44b9ada4a940d3cb25fd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.312207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1399/2773] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:50:00 2022] DELETE /identity/v3/users/af26d817433a44b9ada4a940d3cb25fd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:50:00.635605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:50:00.637234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:00.652316 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acef30-28a0-4b02-85ac-005d73f73bed admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:50:00.712491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:50:00.714324 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:00.717532 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:50:00.718536 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:00.718849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:00.719130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] PATH_INFO: `/v3/projects/615f19017a4f4de999639cc12d36ed0d` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:00.720336 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615f19017a4f4de999639cc12d36ed0d)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:50:00.722269 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:50:00.763605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:50:00.771804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d371a-0687-4334-9c6f-d44baba43aca admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:50:00.773997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615f19017a4f4de999639cc12d36ed0d', 'request_id': 'req-01de8cda-ecec-4624-921d-32a0bf8e18e3'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.780783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615f19017a4f4de999639cc12d36ed0d', 'request_id': 'req-01de8cda-ecec-4624-921d-32a0bf8e18e3'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.786625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615f19017a4f4de999639cc12d36ed0d', 'request_id': 'req-01de8cda-ecec-4624-921d-32a0bf8e18e3'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.800356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] The token cache is being invalidate because project 615f19017a4f4de999639cc12d36ed0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:50:00.800869 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.801551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.803409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1375/2774] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:50:00 2022] DELETE /identity/v3/projects/615f19017a4f4de999639cc12d36ed0d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:50:00.807297 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:50:00.808640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:00.817086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:50:00.823976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5b95e-f172-4795-9aae-3e271ee226ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:50:00.824894 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5b95e-f172-4795-9aae-3e271ee226ed service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:00.825535 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5b95e-f172-4795-9aae-3e271ee226ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:00.825856 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5b95e-f172-4795-9aae-3e271ee226ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:00.827920 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4aa5b95e-f172-4795-9aae-3e271ee226ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:00.874700 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de8cda-ecec-4624-921d-32a0bf8e18e3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:50:00.879473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:50:00.880490 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:00.880777 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:00.881054 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] PATH_INFO: `/v3/users/9aeeb8ddf1a149968ccac8e679080d46` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:00.882270 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aeeb8ddf1a149968ccac8e679080d46)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:50:00.884245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:50:00.894028 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5b95e-f172-4795-9aae-3e271ee226ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:50:00.895644 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa5b95e-f172-4795-9aae-3e271ee226ed service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:50:00.905665 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1400/2775] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:50:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:50:00.932772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aeeb8ddf1a149968ccac8e679080d46'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.933350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aeeb8ddf1a149968ccac8e679080d46'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.946334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aeeb8ddf1a149968ccac8e679080d46'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.946334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aeeb8ddf1a149968ccac8e679080d46'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.954769 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aeeb8ddf1a149968ccac8e679080d46'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:00.968548 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1376/2776] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:50:00 2022] DELETE /identity/v3/users/9aeeb8ddf1a149968ccac8e679080d46 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:50:01.246510 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5b95e-f172-4795-9aae-3e271ee226ed service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:50:01.257617 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5b95e-f172-4795-9aae-3e271ee226ed service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:50:01.336117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:50:01.336994 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:01.337275 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:01.337545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] PATH_INFO: `/v3/projects/0934449a239b482b916556d63430ed33` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:01.339107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0934449a239b482b916556d63430ed33)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:50:01.341165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:50:01.390352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0934449a239b482b916556d63430ed33', 'request_id': 'req-f61b285a-be3e-47b9-857f-a6e1d46319ad'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:01.396990 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0934449a239b482b916556d63430ed33', 'request_id': 'req-f61b285a-be3e-47b9-857f-a6e1d46319ad'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:01.402819 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0934449a239b482b916556d63430ed33', 'request_id': 'req-f61b285a-be3e-47b9-857f-a6e1d46319ad'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:01.416515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] The token cache is being invalidate because project 0934449a239b482b916556d63430ed33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:50:01.416797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:01.417421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:01.419016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1401/2777] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:50:01 2022] DELETE /identity/v3/projects/0934449a239b482b916556d63430ed33 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:50:07.049199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:50:07.050732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:07.064649 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e9339-0bc8-4621-a2b7-2c8a106fddd0 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:50:07.106057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:50:07.123303 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:50:07.124276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:07.124563 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:07.124840 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] PATH_INFO: `/v3/users/1b2337595cbb40608d6bc485889d8bc6` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:07.126131 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b2337595cbb40608d6bc485889d8bc6)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:50:07.128100 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:50:07.186726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b2337595cbb40608d6bc485889d8bc6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.187464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b2337595cbb40608d6bc485889d8bc6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.200830 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b2337595cbb40608d6bc485889d8bc6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.201469 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b2337595cbb40608d6bc485889d8bc6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.210464 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b2337595cbb40608d6bc485889d8bc6'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.223603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1377/2778] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:50:07 2022] DELETE /identity/v3/users/1b2337595cbb40608d6bc485889d8bc6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:50:07.431709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:50:07.433177 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:07.448101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b285a-be3e-47b9-857f-a6e1d46319ad admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:50:07.482797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:50:07.514781 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:50:07.515648 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:07.515924 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:07.516212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] PATH_INFO: `/v3/projects/fa6da3e4a87442299d21f3b55dfe3e9f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:07.517414 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa6da3e4a87442299d21f3b55dfe3e9f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:50:07.519448 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:50:07.568893 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa6da3e4a87442299d21f3b55dfe3e9f', 'request_id': 'req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.574973 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa6da3e4a87442299d21f3b55dfe3e9f', 'request_id': 'req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.580799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa6da3e4a87442299d21f3b55dfe3e9f', 'request_id': 'req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.595526 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] The token cache is being invalidate because project fa6da3e4a87442299d21f3b55dfe3e9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:50:07.595810 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.596481 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.598115 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1402/2779] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:50:07 2022] DELETE /identity/v3/projects/fa6da3e4a87442299d21f3b55dfe3e9f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:50:07.602606 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:50:07.604324 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:07.617978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415bc93-e45a-44db-a07b-004a1e509ab2 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:50:07.677439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:50:07.678319 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:07.678600 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:07.678903 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] PATH_INFO: `/v3/users/a0e1a02e9ec142a7a81bc0b3c5f81500` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:07.697011 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e1a02e9ec142a7a81bc0b3c5f81500)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:50:07.698982 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:50:07.750196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e1a02e9ec142a7a81bc0b3c5f81500'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.750905 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e1a02e9ec142a7a81bc0b3c5f81500'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.755624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:50:07.764946 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e1a02e9ec142a7a81bc0b3c5f81500'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.765612 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e1a02e9ec142a7a81bc0b3c5f81500'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.773680 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e1a02e9ec142a7a81bc0b3c5f81500'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:07.785794 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1378/2780] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:50:07 2022] DELETE /identity/v3/users/a0e1a02e9ec142a7a81bc0b3c5f81500 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:50:07.979958 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:50:07.981639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:07.997233 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8be11a-4266-46a9-a9a4-948dfd0f9e37 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:50:08.056868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:50:08.057660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:08.057974 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:08.058357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] PATH_INFO: `/v3/projects/b4362510130a4e99b74fd45c144de17f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:08.069001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4362510130a4e99b74fd45c144de17f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:50:08.070972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:50:08.117938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4362510130a4e99b74fd45c144de17f', 'request_id': 'req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:08.119799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:50:08.125066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4362510130a4e99b74fd45c144de17f', 'request_id': 'req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:08.131811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4362510130a4e99b74fd45c144de17f', 'request_id': 'req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:08.145536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] The token cache is being invalidate because project b4362510130a4e99b74fd45c144de17f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:50:08.145826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:08.146560 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-64f7b72e-ec27-4fdd-a6c1-2abf7cc67cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:50:08.148250 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1403/2781] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:50:07 2022] DELETE /identity/v3/projects/b4362510130a4e99b74fd45c144de17f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:50:17.782618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:DELETE=3,keystone:SELECT=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:50:18.142299 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=5,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:50:21.940752 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:50:21.942434 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:22.003342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fa170-b357-492f-b3c0-194307c1aa76 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:50:22.053749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96481b2-01d8-490f-948f-ccb8a9c0ed0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:50:22.054439 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96481b2-01d8-490f-948f-ccb8a9c0ed0a service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:22.054709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96481b2-01d8-490f-948f-ccb8a9c0ed0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:22.055020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96481b2-01d8-490f-948f-ccb8a9c0ed0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:22.057027 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b96481b2-01d8-490f-948f-ccb8a9c0ed0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:22.127658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96481b2-01d8-490f-948f-ccb8a9c0ed0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:50:22.129107 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96481b2-01d8-490f-948f-ccb8a9c0ed0a service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:50:22.139363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1379/2782] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:50:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:50:28.158709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ff029-393d-4409-9ea9-3adf7b34a9b9 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:50:28.159253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ff029-393d-4409-9ea9-3adf7b34a9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:50:28.159253 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ff029-393d-4409-9ea9-3adf7b34a9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:50:28.177707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-249ff029-393d-4409-9ea9-3adf7b34a9b9 None None] MFA Rules not processed for user `88a1f82bc0924a84849d390f138041a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:50:28.179030 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-249ff029-393d-4409-9ea9-3adf7b34a9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:50:28.186549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1404/2783] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:50:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:50:32.134367 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=68 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:50:38.172208 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:51:26.489529 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96481b2-01d8-490f-948f-ccb8a9c0ed0a service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:51:26.491174 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b96481b2-01d8-490f-948f-ccb8a9c0ed0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:51:26.551146 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96481b2-01d8-490f-948f-ccb8a9c0ed0a service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:51:26.557518 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a1397-21b3-460c-8113-9b0b918e3de1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:51:26.558479 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a1397-21b3-460c-8113-9b0b918e3de1 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:51:26.558837 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a1397-21b3-460c-8113-9b0b918e3de1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:51:26.559204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a1397-21b3-460c-8113-9b0b918e3de1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:51:26.562624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-078a1397-21b3-460c-8113-9b0b918e3de1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:51:26.576803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a1397-21b3-460c-8113-9b0b918e3de1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:51:26.578283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a1397-21b3-460c-8113-9b0b918e3de1 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:51:26.586765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1380/2784] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:51:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:51:36.583204 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=26 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:51:56.887054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ff029-393d-4409-9ea9-3adf7b34a9b9 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:51:56.888562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-249ff029-393d-4409-9ea9-3adf7b34a9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:51:56.902153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ff029-393d-4409-9ea9-3adf7b34a9b9 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:51:56.905888 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00924fa3-61fe-4b14-88c7-acb964e46145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:51:56.906566 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00924fa3-61fe-4b14-88c7-acb964e46145 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:51:56.906855 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00924fa3-61fe-4b14-88c7-acb964e46145 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:51:56.907129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00924fa3-61fe-4b14-88c7-acb964e46145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:51:56.909336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-00924fa3-61fe-4b14-88c7-acb964e46145 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:51:57.011102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00924fa3-61fe-4b14-88c7-acb964e46145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:51:57.012564 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00924fa3-61fe-4b14-88c7-acb964e46145 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:51:57.034614 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1405/2785] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:51:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:52:06.287354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a1397-21b3-460c-8113-9b0b918e3de1 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:06.296990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a1397-21b3-460c-8113-9b0b918e3de1 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:06.300832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd00ed-29fd-4f7b-a08a-4bb82623d49e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:06.301475 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd00ed-29fd-4f7b-a08a-4bb82623d49e service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:06.301765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd00ed-29fd-4f7b-a08a-4bb82623d49e service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:06.302253 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd00ed-29fd-4f7b-a08a-4bb82623d49e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:06.304409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-00fd00ed-29fd-4f7b-a08a-4bb82623d49e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:06.318003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd00ed-29fd-4f7b-a08a-4bb82623d49e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:06.319708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd00ed-29fd-4f7b-a08a-4bb82623d49e service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:06.330726 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1381/2786] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:52:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:52:07.031838 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=46 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:16.326111 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:21.477135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00924fa3-61fe-4b14-88c7-acb964e46145 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:21.478848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-00924fa3-61fe-4b14-88c7-acb964e46145 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:21.493879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00924fa3-61fe-4b14-88c7-acb964e46145 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:21.500088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b74082-715f-4243-880e-758adf64b642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:21.500796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b74082-715f-4243-880e-758adf64b642 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:21.501083 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b74082-715f-4243-880e-758adf64b642 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:21.501316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b74082-715f-4243-880e-758adf64b642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:21.503656 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-15b74082-715f-4243-880e-758adf64b642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:21.518388 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b74082-715f-4243-880e-758adf64b642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:21.520214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b74082-715f-4243-880e-758adf64b642 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:21.538088 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1406/2787] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:52:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:52:26.443897 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd00ed-29fd-4f7b-a08a-4bb82623d49e service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:26.445355 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-00fd00ed-29fd-4f7b-a08a-4bb82623d49e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:26.457455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd00ed-29fd-4f7b-a08a-4bb82623d49e service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:26.461198 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baed80e-61d3-4392-92b9-e16d6f99d227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:26.461875 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baed80e-61d3-4392-92b9-e16d6f99d227 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:26.462156 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baed80e-61d3-4392-92b9-e16d6f99d227 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:26.462436 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baed80e-61d3-4392-92b9-e16d6f99d227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:26.464614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1baed80e-61d3-4392-92b9-e16d6f99d227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:26.530070 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baed80e-61d3-4392-92b9-e16d6f99d227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:26.531531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baed80e-61d3-4392-92b9-e16d6f99d227 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:26.541364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1382/2788] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:52:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:52:29.734425 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b74082-715f-4243-880e-758adf64b642 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:29.736048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-15b74082-715f-4243-880e-758adf64b642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:29.810660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baed80e-61d3-4392-92b9-e16d6f99d227 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:29.812792 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1baed80e-61d3-4392-92b9-e16d6f99d227 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:29.825582 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b74082-715f-4243-880e-758adf64b642 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:29.827071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baed80e-61d3-4392-92b9-e16d6f99d227 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:29.830567 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832def0-1463-43cc-9bb3-d3910dabeea7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:29.831307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832def0-1463-43cc-9bb3-d3910dabeea7 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:29.831587 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832def0-1463-43cc-9bb3-d3910dabeea7 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:29.831690 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130745f-111a-44df-9427-f22240873a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:29.831864 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832def0-1463-43cc-9bb3-d3910dabeea7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:29.832569 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130745f-111a-44df-9427-f22240873a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:29.832850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130745f-111a-44df-9427-f22240873a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:29.833126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130745f-111a-44df-9427-f22240873a61 admin admin] PATH_INFO: `/v3/users/88a1f82bc0924a84849d390f138041a1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:29.833669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-b832def0-1463-43cc-9bb3-d3910dabeea7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:29.834311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130745f-111a-44df-9427-f22240873a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88a1f82bc0924a84849d390f138041a1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:29.836284 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130745f-111a-44df-9427-f22240873a61 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:29.847101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832def0-1463-43cc-9bb3-d3910dabeea7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:29.848735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832def0-1463-43cc-9bb3-d3910dabeea7 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:29.859939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1407/2789] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:52:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:52:29.899789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1130745f-111a-44df-9427-f22240873a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88a1f82bc0924a84849d390f138041a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:29.900472 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1130745f-111a-44df-9427-f22240873a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88a1f82bc0924a84849d390f138041a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:29.913126 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1130745f-111a-44df-9427-f22240873a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88a1f82bc0924a84849d390f138041a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:29.913816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1130745f-111a-44df-9427-f22240873a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88a1f82bc0924a84849d390f138041a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:29.923076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1130745f-111a-44df-9427-f22240873a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88a1f82bc0924a84849d390f138041a1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:29.935580 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1383/2790] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:52:29 2022] DELETE /identity/v3/users/88a1f82bc0924a84849d390f138041a1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:52:30.116466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832def0-1463-43cc-9bb3-d3910dabeea7 service placement] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:30.127803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832def0-1463-43cc-9bb3-d3910dabeea7 service placement] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:30.192777 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:30.193555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:30.193869 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:30.194149 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] PATH_INFO: `/v3/projects/7faee22e84ba4b8b85f4d4cf6132d83f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:30.195369 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7faee22e84ba4b8b85f4d4cf6132d83f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:30.197291 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:30.248221 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7faee22e84ba4b8b85f4d4cf6132d83f', 'request_id': 'req-81a0c66a-b54e-4284-9da3-a29c60da3317'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:30.254349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7faee22e84ba4b8b85f4d4cf6132d83f', 'request_id': 'req-81a0c66a-b54e-4284-9da3-a29c60da3317'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:30.260180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7faee22e84ba4b8b85f4d4cf6132d83f', 'request_id': 'req-81a0c66a-b54e-4284-9da3-a29c60da3317'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:30.275538 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] The token cache is being invalidate because project 7faee22e84ba4b8b85f4d4cf6132d83f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:52:30.275825 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:30.276443 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-81a0c66a-b54e-4284-9da3-a29c60da3317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:30.278002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1408/2791] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:52:30 2022] DELETE /identity/v3/projects/7faee22e84ba4b8b85f4d4cf6132d83f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:52:30.283309 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa4c34-3956-482a-a4aa-525ac0e4def7 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:30.283602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa4c34-3956-482a-a4aa-525ac0e4def7 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:30.283880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa4c34-3956-482a-a4aa-525ac0e4def7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:30.302161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-c4fa4c34-3956-482a-a4aa-525ac0e4def7 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:30.303410 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-c4fa4c34-3956-482a-a4aa-525ac0e4def7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:30.370705 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1384/2792] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:30.380092 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3a9b7-69a6-48a1-94bd-7420c17e4267 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:30.380416 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3a9b7-69a6-48a1-94bd-7420c17e4267 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:30.380802 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3a9b7-69a6-48a1-94bd-7420c17e4267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:30.398064 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:30.404523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-50f3a9b7-69a6-48a1-94bd-7420c17e4267 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:30.406618 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-50f3a9b7-69a6-48a1-94bd-7420c17e4267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:30.415607 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1409/2793] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:30.420243 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa4c34-3956-482a-a4aa-525ac0e4def7 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:30.430573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa4c34-3956-482a-a4aa-525ac0e4def7 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:30.436445 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06074b-c611-47c5-bb1c-d27375f05355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:30.437135 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06074b-c611-47c5-bb1c-d27375f05355 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:30.437414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06074b-c611-47c5-bb1c-d27375f05355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:30.437692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06074b-c611-47c5-bb1c-d27375f05355 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:30.438427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06074b-c611-47c5-bb1c-d27375f05355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:52:30.438775 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06074b-c611-47c5-bb1c-d27375f05355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:30.440408 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06074b-c611-47c5-bb1c-d27375f05355 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:30.441935 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:30.455186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1385/2794] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:52:30.459409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3a9b7-69a6-48a1-94bd-7420c17e4267 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:30.469738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3a9b7-69a6-48a1-94bd-7420c17e4267 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:30.474986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396ffc8-97a3-4649-871e-f9ba75f05602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:30.475642 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396ffc8-97a3-4649-871e-f9ba75f05602 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:30.475947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396ffc8-97a3-4649-871e-f9ba75f05602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:30.476219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396ffc8-97a3-4649-871e-f9ba75f05602 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:30.476909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396ffc8-97a3-4649-871e-f9ba75f05602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:30.478870 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396ffc8-97a3-4649-871e-f9ba75f05602 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:30.524950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1410/2795] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:52:30 2022] POST /identity/v3/projects => generated 401 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:52:30.531273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06074b-c611-47c5-bb1c-d27375f05355 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:30.541533 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06074b-c611-47c5-bb1c-d27375f05355 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:30.601380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375c840-effa-4d02-960a-4c509f66ce85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:30.602053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375c840-effa-4d02-960a-4c509f66ce85 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:30.602342 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375c840-effa-4d02-960a-4c509f66ce85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:30.602622 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375c840-effa-4d02-960a-4c509f66ce85 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:30.603380 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375c840-effa-4d02-960a-4c509f66ce85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:30.605312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375c840-effa-4d02-960a-4c509f66ce85 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:30.632871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1386/2796] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:52:30 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:52:30.637296 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396ffc8-97a3-4649-871e-f9ba75f05602 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:30.648673 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396ffc8-97a3-4649-871e-f9ba75f05602 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:30.654983 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c90b04-0ac8-4cfc-baac-7336e3f26200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:30.655656 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c90b04-0ac8-4cfc-baac-7336e3f26200 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:30.655938 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c90b04-0ac8-4cfc-baac-7336e3f26200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:30.656196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c90b04-0ac8-4cfc-baac-7336e3f26200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:30.656934 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c90b04-0ac8-4cfc-baac-7336e3f26200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:30.658549 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c90b04-0ac8-4cfc-baac-7336e3f26200 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:30.669129 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1411/2797] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:52:30.673323 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375c840-effa-4d02-960a-4c509f66ce85 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:30.683843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375c840-effa-4d02-960a-4c509f66ce85 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:30.688862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41def7d9-d6c1-4939-9059-a5261afa6fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:30.689531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41def7d9-d6c1-4939-9059-a5261afa6fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:30.689819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41def7d9-d6c1-4939-9059-a5261afa6fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:30.690098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41def7d9-d6c1-4939-9059-a5261afa6fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:30.690879 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41def7d9-d6c1-4939-9059-a5261afa6fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:30.692493 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41def7d9-d6c1-4939-9059-a5261afa6fc2 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:30.702437 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1387/2798] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:52:30.706609 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c90b04-0ac8-4cfc-baac-7336e3f26200 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:30.718206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c90b04-0ac8-4cfc-baac-7336e3f26200 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:30.723280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faee320-d9d4-4d3f-b430-8d339a27a809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:30.724191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faee320-d9d4-4d3f-b430-8d339a27a809 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:30.724468 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faee320-d9d4-4d3f-b430-8d339a27a809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:30.724738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faee320-d9d4-4d3f-b430-8d339a27a809 admin admin] PATH_INFO: `/v3/projects/b72868b474284c248aa5b1ca7a7bd6dc/users/cfe47db06ca84a00b05fcd50b82ec16c/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:30.739461 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faee320-d9d4-4d3f-b430-8d339a27a809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72868b474284c248aa5b1ca7a7bd6dc, user_id=cfe47db06ca84a00b05fcd50b82ec16c, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:30.742105 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faee320-d9d4-4d3f-b430-8d339a27a809 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:30.758035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1412/2799] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:52:30 2022] PUT /identity/v3/projects/b72868b474284c248aa5b1ca7a7bd6dc/users/cfe47db06ca84a00b05fcd50b82ec16c/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:30.763566 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cf8a1-0f39-4fed-9a4e-4032e7a10a5b None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:30.763971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cf8a1-0f39-4fed-9a4e-4032e7a10a5b None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:30.764282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cf8a1-0f39-4fed-9a4e-4032e7a10a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:30.782427 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-e18cf8a1-0f39-4fed-9a4e-4032e7a10a5b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:30.783664 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-e18cf8a1-0f39-4fed-9a4e-4032e7a10a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:30.850213 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1388/2800] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:30.860157 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faee320-d9d4-4d3f-b430-8d339a27a809 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:30.862133 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-2faee320-d9d4-4d3f-b430-8d339a27a809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:30.920423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faee320-d9d4-4d3f-b430-8d339a27a809 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:30.924044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedf4cd-dd65-483b-8d28-351be819076d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:30.924715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedf4cd-dd65-483b-8d28-351be819076d service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:30.924963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedf4cd-dd65-483b-8d28-351be819076d service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:30.925251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedf4cd-dd65-483b-8d28-351be819076d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:30.935020 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedf4cd-dd65-483b-8d28-351be819076d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:30.936488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bedf4cd-dd65-483b-8d28-351be819076d service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:30.947180 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1413/2801] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:52:31.565695 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2d1ac-1d1c-451d-8c7d-a365e7a6119c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:31.566035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2d1ac-1d1c-451d-8c7d-a365e7a6119c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:31.566292 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2d1ac-1d1c-451d-8c7d-a365e7a6119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:31.595299 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-76b2d1ac-1d1c-451d-8c7d-a365e7a6119c None None] MFA Rules not processed for user `cfe47db06ca84a00b05fcd50b82ec16c`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:31.596521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-76b2d1ac-1d1c-451d-8c7d-a365e7a6119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:31.666827 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1389/2802] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:31.671521 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedf4cd-dd65-483b-8d28-351be819076d service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:31.682477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bedf4cd-dd65-483b-8d28-351be819076d service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:31.688293 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31f1ff-4020-40ca-be95-0ce9afc39cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:31.689035 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31f1ff-4020-40ca-be95-0ce9afc39cfd admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:31.689411 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31f1ff-4020-40ca-be95-0ce9afc39cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:31.689752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31f1ff-4020-40ca-be95-0ce9afc39cfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:31.690527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31f1ff-4020-40ca-be95-0ce9afc39cfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:31.692544 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31f1ff-4020-40ca-be95-0ce9afc39cfd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:31.724885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1414/2803] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:52:31 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:52:31.730843 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2d1ac-1d1c-451d-8c7d-a365e7a6119c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:31.736445 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:31.743853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2d1ac-1d1c-451d-8c7d-a365e7a6119c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:31.802164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c8185-422e-4078-aa84-4b4b66d927d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:31.802865 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c8185-422e-4078-aa84-4b4b66d927d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:31.803147 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c8185-422e-4078-aa84-4b4b66d927d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:31.803425 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c8185-422e-4078-aa84-4b4b66d927d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:31.804112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c8185-422e-4078-aa84-4b4b66d927d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:31.806034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c8185-422e-4078-aa84-4b4b66d927d8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:31.829986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1390/2804] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:52:31 2022] POST /identity/v3/users => generated 309 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:52:31.836052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31f1ff-4020-40ca-be95-0ce9afc39cfd admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:31.852729 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31f1ff-4020-40ca-be95-0ce9afc39cfd admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:31.860399 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bc7be-124f-44ba-950e-7cc6bb57ba57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:31.861222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bc7be-124f-44ba-950e-7cc6bb57ba57 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:31.861506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bc7be-124f-44ba-950e-7cc6bb57ba57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:31.861817 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bc7be-124f-44ba-950e-7cc6bb57ba57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:31.862562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bc7be-124f-44ba-950e-7cc6bb57ba57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:31.864239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bc7be-124f-44ba-950e-7cc6bb57ba57 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:31.876435 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1415/2805] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:52:31.880880 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c8185-422e-4078-aa84-4b4b66d927d8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:31.892199 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c8185-422e-4078-aa84-4b4b66d927d8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:31.897161 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf727bb-d52c-4263-94c8-a97631d8b39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:31.898058 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf727bb-d52c-4263-94c8-a97631d8b39c admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:31.898340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf727bb-d52c-4263-94c8-a97631d8b39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:31.898614 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf727bb-d52c-4263-94c8-a97631d8b39c admin admin] PATH_INFO: `/v3/projects/eea8e68ecda14873bb4fd74942f59293/users/28a149bd2fce4b9590e613e18c5a0f70/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:31.918382 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf727bb-d52c-4263-94c8-a97631d8b39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea8e68ecda14873bb4fd74942f59293, user_id=28a149bd2fce4b9590e613e18c5a0f70, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:31.922413 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf727bb-d52c-4263-94c8-a97631d8b39c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:31.941310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1391/2806] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:52:31 2022] PUT /identity/v3/projects/eea8e68ecda14873bb4fd74942f59293/users/28a149bd2fce4b9590e613e18c5a0f70/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:31.945354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bc7be-124f-44ba-950e-7cc6bb57ba57 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:31.955950 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bc7be-124f-44ba-950e-7cc6bb57ba57 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:32.017824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c5985-6f27-456a-83b9-cff6911ac8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:32.018446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c5985-6f27-456a-83b9-cff6911ac8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:32.018755 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c5985-6f27-456a-83b9-cff6911ac8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:32.019054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c5985-6f27-456a-83b9-cff6911ac8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:32.020131 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c5985-6f27-456a-83b9-cff6911ac8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:32.022629 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c5985-6f27-456a-83b9-cff6911ac8d4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:32.030008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:32.035898 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1416/2807] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:52:32.040189 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf727bb-d52c-4263-94c8-a97631d8b39c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:32.052034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf727bb-d52c-4263-94c8-a97631d8b39c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:32.057407 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490b9f9-21e0-41fe-b4ac-e000e8976bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:32.058300 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490b9f9-21e0-41fe-b4ac-e000e8976bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:32.058577 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490b9f9-21e0-41fe-b4ac-e000e8976bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:32.058885 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490b9f9-21e0-41fe-b4ac-e000e8976bbf admin admin] PATH_INFO: `/v3/projects/eea8e68ecda14873bb4fd74942f59293/users/28a149bd2fce4b9590e613e18c5a0f70/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:32.061208 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490b9f9-21e0-41fe-b4ac-e000e8976bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea8e68ecda14873bb4fd74942f59293, user_id=28a149bd2fce4b9590e613e18c5a0f70, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:32.063890 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490b9f9-21e0-41fe-b4ac-e000e8976bbf admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:32.075059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1392/2808] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:52:32 2022] PUT /identity/v3/projects/eea8e68ecda14873bb4fd74942f59293/users/28a149bd2fce4b9590e613e18c5a0f70/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:32.744538 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351a06b-3a42-44c2-a50b-0974c7207e87 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:32.744829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351a06b-3a42-44c2-a50b-0974c7207e87 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:32.745101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351a06b-3a42-44c2-a50b-0974c7207e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:32.774906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4351a06b-3a42-44c2-a50b-0974c7207e87 None None] MFA Rules not processed for user `28a149bd2fce4b9590e613e18c5a0f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:32.776193 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4351a06b-3a42-44c2-a50b-0974c7207e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:32.855037 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1417/2809] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:32 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:33.915167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490b9f9-21e0-41fe-b4ac-e000e8976bbf admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:33.980725 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490b9f9-21e0-41fe-b4ac-e000e8976bbf admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:33.985852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:33.986737 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:33.987053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:33.987321 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] PATH_INFO: `/v3/users/cfe47db06ca84a00b05fcd50b82ec16c` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:33.988549 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe47db06ca84a00b05fcd50b82ec16c)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:33.990526 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:34.038203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe47db06ca84a00b05fcd50b82ec16c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.038919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe47db06ca84a00b05fcd50b82ec16c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.045990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:34.051988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe47db06ca84a00b05fcd50b82ec16c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.052685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe47db06ca84a00b05fcd50b82ec16c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.062406 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe47db06ca84a00b05fcd50b82ec16c'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.075091 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1393/2810] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:52:33 2022] DELETE /identity/v3/users/cfe47db06ca84a00b05fcd50b82ec16c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:34.252350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351a06b-3a42-44c2-a50b-0974c7207e87 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:34.263423 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351a06b-3a42-44c2-a50b-0974c7207e87 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:34.324797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:34.325703 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:34.325965 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:34.326264 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] PATH_INFO: `/v3/projects/b72868b474284c248aa5b1ca7a7bd6dc` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:34.327606 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b72868b474284c248aa5b1ca7a7bd6dc)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:34.329506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:34.380066 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b72868b474284c248aa5b1ca7a7bd6dc', 'request_id': 'req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.386352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b72868b474284c248aa5b1ca7a7bd6dc', 'request_id': 'req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.392314 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b72868b474284c248aa5b1ca7a7bd6dc', 'request_id': 'req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.406463 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] The token cache is being invalidate because project b72868b474284c248aa5b1ca7a7bd6dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:52:34.406778 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.407405 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.408985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1418/2811] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:52:34 2022] DELETE /identity/v3/projects/b72868b474284c248aa5b1ca7a7bd6dc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:34.413115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:34.414714 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:34.428687 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbe67a-f5ed-47b6-85f0-b5649dc51ce8 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:34.486530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf16c8-85a1-4ae6-a3ad-45aefbf1c1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:34.487467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf16c8-85a1-4ae6-a3ad-45aefbf1c1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:34.487750 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf16c8-85a1-4ae6-a3ad-45aefbf1c1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:34.488025 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf16c8-85a1-4ae6-a3ad-45aefbf1c1d8 admin admin] PATH_INFO: `/v3/users/28a149bd2fce4b9590e613e18c5a0f70` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:34.489365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf16c8-85a1-4ae6-a3ad-45aefbf1c1d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28a149bd2fce4b9590e613e18c5a0f70)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:34.491354 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf16c8-85a1-4ae6-a3ad-45aefbf1c1d8 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:34.542766 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c8bf16c8-85a1-4ae6-a3ad-45aefbf1c1d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28a149bd2fce4b9590e613e18c5a0f70'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.543327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c8bf16c8-85a1-4ae6-a3ad-45aefbf1c1d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28a149bd2fce4b9590e613e18c5a0f70'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.555511 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c8bf16c8-85a1-4ae6-a3ad-45aefbf1c1d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28a149bd2fce4b9590e613e18c5a0f70'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.556109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c8bf16c8-85a1-4ae6-a3ad-45aefbf1c1d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28a149bd2fce4b9590e613e18c5a0f70'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.565730 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-c8bf16c8-85a1-4ae6-a3ad-45aefbf1c1d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28a149bd2fce4b9590e613e18c5a0f70'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.582562 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1394/2812] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:52:34 2022] DELETE /identity/v3/users/28a149bd2fce4b9590e613e18c5a0f70 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:34.776014 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:34.777524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:34.793652 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e28b66-d6e7-4d9c-ac3c-5bd9e6696cf3 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:34.806592 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:34.853249 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:34.854117 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:34.854398 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:34.854664 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] PATH_INFO: `/v3/projects/eea8e68ecda14873bb4fd74942f59293` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:34.855906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea8e68ecda14873bb4fd74942f59293)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:34.857907 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:34.909823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea8e68ecda14873bb4fd74942f59293', 'request_id': 'req-a1d52c77-3994-4e61-94c9-0dc607f818e4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.915928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea8e68ecda14873bb4fd74942f59293', 'request_id': 'req-a1d52c77-3994-4e61-94c9-0dc607f818e4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.921846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea8e68ecda14873bb4fd74942f59293', 'request_id': 'req-a1d52c77-3994-4e61-94c9-0dc607f818e4'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.937205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] The token cache is being invalidate because project eea8e68ecda14873bb4fd74942f59293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:52:34.937497 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.938165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-a1d52c77-3994-4e61-94c9-0dc607f818e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:34.939782 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1419/2813] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:52:34 2022] DELETE /identity/v3/projects/eea8e68ecda14873bb4fd74942f59293 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:34.944901 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb819c9-4910-42eb-b4e0-9bcf2b597054 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:34.945364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb819c9-4910-42eb-b4e0-9bcf2b597054 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:34.945474 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb819c9-4910-42eb-b4e0-9bcf2b597054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:34.965465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5cb819c9-4910-42eb-b4e0-9bcf2b597054 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:34.966681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5cb819c9-4910-42eb-b4e0-9bcf2b597054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:35.016513 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:35.035024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1395/2814] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:35.042966 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ceb6-7d87-4571-846b-aa15e9b32b3e None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.043260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ceb6-7d87-4571-846b-aa15e9b32b3e None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.043541 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ceb6-7d87-4571-846b-aa15e9b32b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.064829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-7996ceb6-7d87-4571-846b-aa15e9b32b3e None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:35.065979 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7996ceb6-7d87-4571-846b-aa15e9b32b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:35.074048 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1420/2815] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:35.078835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb819c9-4910-42eb-b4e0-9bcf2b597054 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.089521 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb819c9-4910-42eb-b4e0-9bcf2b597054 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.095409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d573dd-d534-4359-bc52-aa55c03d3775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.096048 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d573dd-d534-4359-bc52-aa55c03d3775 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.096335 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d573dd-d534-4359-bc52-aa55c03d3775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.096604 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d573dd-d534-4359-bc52-aa55c03d3775 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.097345 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d573dd-d534-4359-bc52-aa55c03d3775 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:52:35.097654 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d573dd-d534-4359-bc52-aa55c03d3775 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.099296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d573dd-d534-4359-bc52-aa55c03d3775 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.113124 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1396/2816] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 08:52:35.117686 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996ceb6-7d87-4571-846b-aa15e9b32b3e None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.127816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996ceb6-7d87-4571-846b-aa15e9b32b3e None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.133818 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d57b06-b457-469b-9097-1f14af157589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.134460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d57b06-b457-469b-9097-1f14af157589 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.134776 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d57b06-b457-469b-9097-1f14af157589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.135054 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d57b06-b457-469b-9097-1f14af157589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.135757 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d57b06-b457-469b-9097-1f14af157589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.137688 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d57b06-b457-469b-9097-1f14af157589 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.169401 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1421/2817] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:52:35 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:52:35.171116 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d573dd-d534-4359-bc52-aa55c03d3775 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.182024 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d573dd-d534-4359-bc52-aa55c03d3775 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.243037 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09167b3b-a7ba-440d-9597-0075880a8536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.243708 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09167b3b-a7ba-440d-9597-0075880a8536 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.243990 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09167b3b-a7ba-440d-9597-0075880a8536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.244261 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09167b3b-a7ba-440d-9597-0075880a8536 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.244954 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09167b3b-a7ba-440d-9597-0075880a8536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.246900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09167b3b-a7ba-440d-9597-0075880a8536 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.269183 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1397/2818] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:52:35 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:52:35.273729 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d57b06-b457-469b-9097-1f14af157589 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.283947 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d57b06-b457-469b-9097-1f14af157589 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.289440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bba5c-3a75-452e-8744-4eb4a26e6a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.290147 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bba5c-3a75-452e-8744-4eb4a26e6a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.290428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bba5c-3a75-452e-8744-4eb4a26e6a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.290724 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bba5c-3a75-452e-8744-4eb4a26e6a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.291481 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bba5c-3a75-452e-8744-4eb4a26e6a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.293102 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bba5c-3a75-452e-8744-4eb4a26e6a64 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.304201 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1422/2819] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:52:35.308283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09167b3b-a7ba-440d-9597-0075880a8536 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.320578 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09167b3b-a7ba-440d-9597-0075880a8536 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.325665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8322a86-7924-4b91-9f32-f6d91794ab17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.326372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8322a86-7924-4b91-9f32-f6d91794ab17 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.326663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8322a86-7924-4b91-9f32-f6d91794ab17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.327076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8322a86-7924-4b91-9f32-f6d91794ab17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.327850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8322a86-7924-4b91-9f32-f6d91794ab17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.329507 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8322a86-7924-4b91-9f32-f6d91794ab17 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.340318 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1398/2820] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:52:35.344626 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bba5c-3a75-452e-8744-4eb4a26e6a64 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.355094 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bba5c-3a75-452e-8744-4eb4a26e6a64 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.361047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417aeac-1287-4c9f-8722-18aabac2af67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.361939 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417aeac-1287-4c9f-8722-18aabac2af67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.362217 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417aeac-1287-4c9f-8722-18aabac2af67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.362485 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417aeac-1287-4c9f-8722-18aabac2af67 admin admin] PATH_INFO: `/v3/projects/de1b73bc078b4dd6a95a7531bb32d46d/users/63fa54406de3485283af834d572ceeb4/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.378682 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417aeac-1287-4c9f-8722-18aabac2af67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1b73bc078b4dd6a95a7531bb32d46d, user_id=63fa54406de3485283af834d572ceeb4, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.381391 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417aeac-1287-4c9f-8722-18aabac2af67 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.396162 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1423/2821] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:52:35 2022] PUT /identity/v3/projects/de1b73bc078b4dd6a95a7531bb32d46d/users/63fa54406de3485283af834d572ceeb4/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:35.406458 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246e8aa-46cd-467d-8299-1e6ba16e7cf6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.406803 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246e8aa-46cd-467d-8299-1e6ba16e7cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.407112 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246e8aa-46cd-467d-8299-1e6ba16e7cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.434742 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b246e8aa-46cd-467d-8299-1e6ba16e7cf6 None None] MFA Rules not processed for user `63fa54406de3485283af834d572ceeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:35.435997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b246e8aa-46cd-467d-8299-1e6ba16e7cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:35.510852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1399/2822] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:35 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:35.515837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417aeac-1287-4c9f-8722-18aabac2af67 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.562763 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:35.583015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417aeac-1287-4c9f-8722-18aabac2af67 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.588868 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb79a72-c170-44d6-96be-d037f61c6ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.589537 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb79a72-c170-44d6-96be-d037f61c6ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.589814 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb79a72-c170-44d6-96be-d037f61c6ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.590082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb79a72-c170-44d6-96be-d037f61c6ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.590803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb79a72-c170-44d6-96be-d037f61c6ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.592715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb79a72-c170-44d6-96be-d037f61c6ca1 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.621270 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1424/2823] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:52:35 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:52:35.625610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246e8aa-46cd-467d-8299-1e6ba16e7cf6 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.637603 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246e8aa-46cd-467d-8299-1e6ba16e7cf6 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.684313 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:35.697648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f728f9-b262-435e-b455-ad1fbb77541c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.698322 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f728f9-b262-435e-b455-ad1fbb77541c admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.698601 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f728f9-b262-435e-b455-ad1fbb77541c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.698914 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f728f9-b262-435e-b455-ad1fbb77541c admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.699613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f728f9-b262-435e-b455-ad1fbb77541c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.701574 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f728f9-b262-435e-b455-ad1fbb77541c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.724234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1400/2824] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:52:35 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:52:35.728329 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb79a72-c170-44d6-96be-d037f61c6ca1 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.738986 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb79a72-c170-44d6-96be-d037f61c6ca1 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.743877 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b92d80-970e-4bf9-be2d-7bf7c1b2930c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.744579 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92d80-970e-4bf9-be2d-7bf7c1b2930c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.744862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92d80-970e-4bf9-be2d-7bf7c1b2930c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.745140 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b92d80-970e-4bf9-be2d-7bf7c1b2930c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.745872 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b92d80-970e-4bf9-be2d-7bf7c1b2930c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.747530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b92d80-970e-4bf9-be2d-7bf7c1b2930c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.758357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1425/2825] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:52:35.762613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f728f9-b262-435e-b455-ad1fbb77541c admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.773165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f728f9-b262-435e-b455-ad1fbb77541c admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.778807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ab67e-02fc-477e-8006-ab2e8555ab69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.779850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ab67e-02fc-477e-8006-ab2e8555ab69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.780167 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ab67e-02fc-477e-8006-ab2e8555ab69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.780473 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ab67e-02fc-477e-8006-ab2e8555ab69 admin admin] PATH_INFO: `/v3/projects/445743d086414dedb16ec4d9c434812f/users/df95c0a864c44a36a06412c3a45edcd2/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.797486 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ab67e-02fc-477e-8006-ab2e8555ab69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=445743d086414dedb16ec4d9c434812f, user_id=df95c0a864c44a36a06412c3a45edcd2, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.800201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ab67e-02fc-477e-8006-ab2e8555ab69 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.815412 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1401/2826] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:52:35 2022] PUT /identity/v3/projects/445743d086414dedb16ec4d9c434812f/users/df95c0a864c44a36a06412c3a45edcd2/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:35.819440 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b92d80-970e-4bf9-be2d-7bf7c1b2930c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.829795 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b92d80-970e-4bf9-be2d-7bf7c1b2930c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.890114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97330e85-2fb3-4285-a41a-ab130229efe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.890811 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97330e85-2fb3-4285-a41a-ab130229efe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.891076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97330e85-2fb3-4285-a41a-ab130229efe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.891382 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97330e85-2fb3-4285-a41a-ab130229efe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.892110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97330e85-2fb3-4285-a41a-ab130229efe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.893756 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97330e85-2fb3-4285-a41a-ab130229efe4 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.903961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1426/2827] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:52:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:52:35.908452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ab67e-02fc-477e-8006-ab2e8555ab69 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:35.918879 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ab67e-02fc-477e-8006-ab2e8555ab69 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:35.923962 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d464d-a832-4099-bb67-7d7bf3225917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:35.924852 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d464d-a832-4099-bb67-7d7bf3225917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.925138 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d464d-a832-4099-bb67-7d7bf3225917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.925414 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d464d-a832-4099-bb67-7d7bf3225917 admin admin] PATH_INFO: `/v3/projects/445743d086414dedb16ec4d9c434812f/users/df95c0a864c44a36a06412c3a45edcd2/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.927468 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d464d-a832-4099-bb67-7d7bf3225917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=445743d086414dedb16ec4d9c434812f, user_id=df95c0a864c44a36a06412c3a45edcd2, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:35.930092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d464d-a832-4099-bb67-7d7bf3225917 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:35.940677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1402/2828] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:52:35 2022] PUT /identity/v3/projects/445743d086414dedb16ec4d9c434812f/users/df95c0a864c44a36a06412c3a45edcd2/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:35.952052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75087626-4ce0-44c5-bcac-3558fcb4e516 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:35.952390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75087626-4ce0-44c5-bcac-3558fcb4e516 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:35.952736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75087626-4ce0-44c5-bcac-3558fcb4e516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:35.982471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-75087626-4ce0-44c5-bcac-3558fcb4e516 None None] MFA Rules not processed for user `df95c0a864c44a36a06412c3a45edcd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:35.983803 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-75087626-4ce0-44c5-bcac-3558fcb4e516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:36.065107 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1427/2829] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:35 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:36.077180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d464d-a832-4099-bb67-7d7bf3225917 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:36.079233 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-0b7d464d-a832-4099-bb67-7d7bf3225917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:36.146296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d464d-a832-4099-bb67-7d7bf3225917 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:36.150514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e7da6-a639-4fea-80dd-cf5e1cd1407f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:36.151201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7da6-a639-4fea-80dd-cf5e1cd1407f service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:36.151481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7da6-a639-4fea-80dd-cf5e1cd1407f service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:36.151749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7da6-a639-4fea-80dd-cf5e1cd1407f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:36.161739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e7da6-a639-4fea-80dd-cf5e1cd1407f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:36.163241 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e7da6-a639-4fea-80dd-cf5e1cd1407f service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:36.173272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1403/2830] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:52:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:52:36.260280 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75087626-4ce0-44c5-bcac-3558fcb4e516 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:36.272698 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75087626-4ce0-44c5-bcac-3558fcb4e516 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:36.276768 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c879d-28a1-4346-97cc-64f8b2d8bb83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:36.277426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c879d-28a1-4346-97cc-64f8b2d8bb83 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:36.277709 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c879d-28a1-4346-97cc-64f8b2d8bb83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:36.278010 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c879d-28a1-4346-97cc-64f8b2d8bb83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:36.313315 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:36.338816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c879d-28a1-4346-97cc-64f8b2d8bb83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:36.340262 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c879d-28a1-4346-97cc-64f8b2d8bb83 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:36.349545 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1428/2831] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:52:36 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:52:46.168672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=48,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:46.345297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=11 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:47.808326 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e7da6-a639-4fea-80dd-cf5e1cd1407f service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:47.873737 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e7da6-a639-4fea-80dd-cf5e1cd1407f service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:47.878935 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:47.879812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:47.880092 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:47.880366 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] PATH_INFO: `/v3/users/63fa54406de3485283af834d572ceeb4` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:47.881608 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=63fa54406de3485283af834d572ceeb4)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:47.883758 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:47.932658 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63fa54406de3485283af834d572ceeb4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:47.933311 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63fa54406de3485283af834d572ceeb4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:47.945334 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63fa54406de3485283af834d572ceeb4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:47.945967 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63fa54406de3485283af834d572ceeb4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:47.954847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63fa54406de3485283af834d572ceeb4'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:47.967859 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1404/2832] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:52:47 2022] DELETE /identity/v3/users/63fa54406de3485283af834d572ceeb4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:47.972675 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f8075-4c1f-4c2f-8e81-f48314e93b23 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:47.973166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f8075-4c1f-4c2f-8e81-f48314e93b23 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:47.973166 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f8075-4c1f-4c2f-8e81-f48314e93b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:47.991669 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-817f8075-4c1f-4c2f-8e81-f48314e93b23 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:47.993079 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-817f8075-4c1f-4c2f-8e81-f48314e93b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:48.054710 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1429/2833] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:48.064767 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:48.066476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:48.124714 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98be61-3530-4901-a5c9-7c241fc0bcbe admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:48.128762 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ec323-3b72-43fa-b038-ff2c8d5d9d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:48.129417 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ec323-3b72-43fa-b038-ff2c8d5d9d17 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:48.129703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ec323-3b72-43fa-b038-ff2c8d5d9d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:48.129985 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ec323-3b72-43fa-b038-ff2c8d5d9d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:48.139651 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ec323-3b72-43fa-b038-ff2c8d5d9d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:48.141101 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ec323-3b72-43fa-b038-ff2c8d5d9d17 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:48.150850 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1405/2834] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:52:48.423638 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f8075-4c1f-4c2f-8e81-f48314e93b23 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:48.425229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-817f8075-4c1f-4c2f-8e81-f48314e93b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:48.438015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f8075-4c1f-4c2f-8e81-f48314e93b23 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:48.443371 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:48.444269 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:48.444559 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:48.444821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] PATH_INFO: `/v3/projects/de1b73bc078b4dd6a95a7531bb32d46d` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:48.446227 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1b73bc078b4dd6a95a7531bb32d46d)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:48.448187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:48.505857 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1b73bc078b4dd6a95a7531bb32d46d', 'request_id': 'req-d965294e-b9da-456e-82c0-60b932c24902'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:48.512240 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1b73bc078b4dd6a95a7531bb32d46d', 'request_id': 'req-d965294e-b9da-456e-82c0-60b932c24902'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:48.518325 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1b73bc078b4dd6a95a7531bb32d46d', 'request_id': 'req-d965294e-b9da-456e-82c0-60b932c24902'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:48.533822 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] The token cache is being invalidate because project de1b73bc078b4dd6a95a7531bb32d46d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:52:48.534104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:48.534796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:48.536478 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1430/2835] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:52:48 2022] DELETE /identity/v3/projects/de1b73bc078b4dd6a95a7531bb32d46d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:48.540717 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ec323-3b72-43fa-b038-ff2c8d5d9d17 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:48.550778 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ec323-3b72-43fa-b038-ff2c8d5d9d17 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:48.569935 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:48.612165 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a7157-998f-48f9-bee8-58667675a08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:48.613043 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a7157-998f-48f9-bee8-58667675a08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:48.613349 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a7157-998f-48f9-bee8-58667675a08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:48.613624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a7157-998f-48f9-bee8-58667675a08c admin admin] PATH_INFO: `/v3/users/df95c0a864c44a36a06412c3a45edcd2` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:48.614900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a7157-998f-48f9-bee8-58667675a08c admin admin] RBAC: Authorizing `identity:delete_user(user_id=df95c0a864c44a36a06412c3a45edcd2)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:48.616855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a7157-998f-48f9-bee8-58667675a08c admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:48.665736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8a6a7157-998f-48f9-bee8-58667675a08c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df95c0a864c44a36a06412c3a45edcd2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:48.666426 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8a6a7157-998f-48f9-bee8-58667675a08c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df95c0a864c44a36a06412c3a45edcd2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:48.678461 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8a6a7157-998f-48f9-bee8-58667675a08c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df95c0a864c44a36a06412c3a45edcd2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:48.679117 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8a6a7157-998f-48f9-bee8-58667675a08c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df95c0a864c44a36a06412c3a45edcd2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:48.688386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-8a6a7157-998f-48f9-bee8-58667675a08c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df95c0a864c44a36a06412c3a45edcd2'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:48.700618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1406/2836] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:52:48 2022] DELETE /identity/v3/users/df95c0a864c44a36a06412c3a45edcd2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:48.981354 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:48.982928 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:48.997342 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965294e-b9da-456e-82c0-60b932c24902 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:49.060251 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:49.061136 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:49.061414 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:49.061683 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] PATH_INFO: `/v3/projects/445743d086414dedb16ec4d9c434812f` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:49.062922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=445743d086414dedb16ec4d9c434812f)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:49.064852 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:49.099392 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:49.122044 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445743d086414dedb16ec4d9c434812f', 'request_id': 'req-21d86c92-d593-4ccf-9e13-7f88d00891fa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:49.128082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445743d086414dedb16ec4d9c434812f', 'request_id': 'req-21d86c92-d593-4ccf-9e13-7f88d00891fa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:49.133674 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445743d086414dedb16ec4d9c434812f', 'request_id': 'req-21d86c92-d593-4ccf-9e13-7f88d00891fa'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:49.148198 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] The token cache is being invalidate because project 445743d086414dedb16ec4d9c434812f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:52:49.148499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:49.149331 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-21d86c92-d593-4ccf-9e13-7f88d00891fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:52:49.151006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1431/2837] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:52:48 2022] DELETE /identity/v3/projects/445743d086414dedb16ec4d9c434812f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:49.156196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913d05-a44f-4402-a649-c421918ba4b3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:49.156692 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913d05-a44f-4402-a649-c421918ba4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:49.156789 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913d05-a44f-4402-a649-c421918ba4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:49.175602 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-75913d05-a44f-4402-a649-c421918ba4b3 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:49.176824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-75913d05-a44f-4402-a649-c421918ba4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:49.237989 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1407/2838] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:49.246226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7586a1-a747-49d6-ae73-9ef4e4397e4b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:49.246515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7586a1-a747-49d6-ae73-9ef4e4397e4b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:49.246829 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7586a1-a747-49d6-ae73-9ef4e4397e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:49.266407 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-bd7586a1-a747-49d6-ae73-9ef4e4397e4b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:49.267660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-bd7586a1-a747-49d6-ae73-9ef4e4397e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:49.277151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1432/2839] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:49.283094 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913d05-a44f-4402-a649-c421918ba4b3 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:49.293350 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913d05-a44f-4402-a649-c421918ba4b3 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:49.298909 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f9b18-cc68-4a48-af22-c1c7b2352ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:49.299575 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f9b18-cc68-4a48-af22-c1c7b2352ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:49.299863 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f9b18-cc68-4a48-af22-c1c7b2352ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:49.300144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f9b18-cc68-4a48-af22-c1c7b2352ab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:49.300889 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f9b18-cc68-4a48-af22-c1c7b2352ab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:52:49.301206 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f9b18-cc68-4a48-af22-c1c7b2352ab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:49.302857 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f9b18-cc68-4a48-af22-c1c7b2352ab1 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:49.315114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1408/2840] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:52:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:52:49.319565 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7586a1-a747-49d6-ae73-9ef4e4397e4b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:49.329395 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7586a1-a747-49d6-ae73-9ef4e4397e4b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:49.334672 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b8165-2b9e-4831-9424-80602ca5e804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:49.335442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b8165-2b9e-4831-9424-80602ca5e804 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:49.335719 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b8165-2b9e-4831-9424-80602ca5e804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:49.335997 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b8165-2b9e-4831-9424-80602ca5e804 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:49.336764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b8165-2b9e-4831-9424-80602ca5e804 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:49.339191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b8165-2b9e-4831-9424-80602ca5e804 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:49.368371 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1433/2841] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:52:49 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:52:49.373369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f9b18-cc68-4a48-af22-c1c7b2352ab1 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:49.388266 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f9b18-cc68-4a48-af22-c1c7b2352ab1 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:49.415715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:49.453771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c491b3-ab50-4b01-a0ff-895199b00c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:49.454451 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c491b3-ab50-4b01-a0ff-895199b00c7f admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:49.454743 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c491b3-ab50-4b01-a0ff-895199b00c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:49.455010 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c491b3-ab50-4b01-a0ff-895199b00c7f admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:49.455709 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c491b3-ab50-4b01-a0ff-895199b00c7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:49.458292 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c491b3-ab50-4b01-a0ff-895199b00c7f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:49.481000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1409/2842] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:52:49 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:52:49.485187 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b8165-2b9e-4831-9424-80602ca5e804 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:49.496761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b8165-2b9e-4831-9424-80602ca5e804 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:49.502859 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12340c-1778-43da-b5ec-7055f51ec57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:49.503528 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12340c-1778-43da-b5ec-7055f51ec57f admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:49.503821 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12340c-1778-43da-b5ec-7055f51ec57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:49.504101 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12340c-1778-43da-b5ec-7055f51ec57f admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:49.504840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12340c-1778-43da-b5ec-7055f51ec57f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:49.506460 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12340c-1778-43da-b5ec-7055f51ec57f admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:49.517920 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1434/2843] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:52:49.522127 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c491b3-ab50-4b01-a0ff-895199b00c7f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:49.533379 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c491b3-ab50-4b01-a0ff-895199b00c7f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:49.539381 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfcf450-09ac-4d3f-bf4b-ab48b3094994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:49.540169 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf450-09ac-4d3f-bf4b-ab48b3094994 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:49.540526 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf450-09ac-4d3f-bf4b-ab48b3094994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:49.540806 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfcf450-09ac-4d3f-bf4b-ab48b3094994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:49.541546 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcf450-09ac-4d3f-bf4b-ab48b3094994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:49.543185 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfcf450-09ac-4d3f-bf4b-ab48b3094994 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:49.553871 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1410/2844] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:52:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:52:49.558219 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12340c-1778-43da-b5ec-7055f51ec57f admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:49.568836 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12340c-1778-43da-b5ec-7055f51ec57f admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:49.574508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fa2ec-d6e6-46a8-ae72-fc3970f6fa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:49.575426 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa2ec-d6e6-46a8-ae72-fc3970f6fa14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:49.575708 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa2ec-d6e6-46a8-ae72-fc3970f6fa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:49.575985 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa2ec-d6e6-46a8-ae72-fc3970f6fa14 admin admin] PATH_INFO: `/v3/projects/a721304960684fda928acfd30cb4434a/users/0eb00537a0704b23a67108e94380aaa7/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:49.595260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fa2ec-d6e6-46a8-ae72-fc3970f6fa14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a721304960684fda928acfd30cb4434a, user_id=0eb00537a0704b23a67108e94380aaa7, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:49.599572 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288fa2ec-d6e6-46a8-ae72-fc3970f6fa14 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:49.616513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1435/2845] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:52:49 2022] PUT /identity/v3/projects/a721304960684fda928acfd30cb4434a/users/0eb00537a0704b23a67108e94380aaa7/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:52:49.628090 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcdf0e-40f9-4fec-8d15-e36d9f209dc8 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:49.628384 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcdf0e-40f9-4fec-8d15-e36d9f209dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:49.628666 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fcdf0e-40f9-4fec-8d15-e36d9f209dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:49.655400 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-d3fcdf0e-40f9-4fec-8d15-e36d9f209dc8 None None] MFA Rules not processed for user `0eb00537a0704b23a67108e94380aaa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:52:49.656611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d3fcdf0e-40f9-4fec-8d15-e36d9f209dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:49.724987 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1411/2846] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:52:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:52:49.737632 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fa2ec-d6e6-46a8-ae72-fc3970f6fa14 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:52:49.739804 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-288fa2ec-d6e6-46a8-ae72-fc3970f6fa14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:52:49.822660 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fa2ec-d6e6-46a8-ae72-fc3970f6fa14 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:52:49.826812 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f394b2-945a-459c-9988-10331d714828 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd09453b5db645d7b03522f91effb8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:52:49.827455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f394b2-945a-459c-9988-10331d714828 service glance] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:52:49.827733 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f394b2-945a-459c-9988-10331d714828 service glance] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:52:49.828000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f394b2-945a-459c-9988-10331d714828 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:52:49.837422 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f394b2-945a-459c-9988-10331d714828 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:52:49.839222 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f394b2-945a-459c-9988-10331d714828 service glance] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:52:49.848207 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1436/2847] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:52:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:52:59.722061 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:52:59.844286 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:DELETE=6,keystone:SELECT=76,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:53:11.778368 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcdf0e-40f9-4fec-8d15-e36d9f209dc8 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:11.852868 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fcdf0e-40f9-4fec-8d15-e36d9f209dc8 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:11.858807 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:11.859696 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:11.860008 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:11.860283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] PATH_INFO: `/v3/users/0eb00537a0704b23a67108e94380aaa7` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:11.861661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eb00537a0704b23a67108e94380aaa7)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:11.863764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:11.912482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eb00537a0704b23a67108e94380aaa7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:53:11.913153 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eb00537a0704b23a67108e94380aaa7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:53:11.927164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eb00537a0704b23a67108e94380aaa7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:53:11.929039 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eb00537a0704b23a67108e94380aaa7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:53:11.937476 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eb00537a0704b23a67108e94380aaa7'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:53:11.951919 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1412/2848] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:53:11 2022] DELETE /identity/v3/users/0eb00537a0704b23a67108e94380aaa7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:53:11.956741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f460738-082b-4e19-8e51-03b48773e654 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:11.957242 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f460738-082b-4e19-8e51-03b48773e654 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:11.957352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f460738-082b-4e19-8e51-03b48773e654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:11.976176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-8f460738-082b-4e19-8e51-03b48773e654 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:53:11.977376 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8f460738-082b-4e19-8e51-03b48773e654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:12.043670 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1437/2849] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:53:12.053618 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:12.055310 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:12.115433 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833a835-266c-47eb-b64b-f1fbc32dd856 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:12.120003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa131c42-b2cc-4874-9dd5-f53d3dd4a5f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:12.120660 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa131c42-b2cc-4874-9dd5-f53d3dd4a5f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:12.121005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa131c42-b2cc-4874-9dd5-f53d3dd4a5f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:12.121286 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa131c42-b2cc-4874-9dd5-f53d3dd4a5f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:12.137282 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa131c42-b2cc-4874-9dd5-f53d3dd4a5f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:12.139565 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa131c42-b2cc-4874-9dd5-f53d3dd4a5f1 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:12.154404 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1413/2850] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:53:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:53:12.465593 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f460738-082b-4e19-8e51-03b48773e654 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:12.467301 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-8f460738-082b-4e19-8e51-03b48773e654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:12.479489 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f460738-082b-4e19-8e51-03b48773e654 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:12.484650 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:12.485534 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:12.485823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:12.486110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] PATH_INFO: `/v3/projects/a721304960684fda928acfd30cb4434a` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:12.487430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a721304960684fda928acfd30cb4434a)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:12.489343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:12.539677 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a721304960684fda928acfd30cb4434a', 'request_id': 'req-c6ccf0c0-c318-4714-9168-8587c849e9dd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:53:12.546503 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a721304960684fda928acfd30cb4434a', 'request_id': 'req-c6ccf0c0-c318-4714-9168-8587c849e9dd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:53:12.552128 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a721304960684fda928acfd30cb4434a', 'request_id': 'req-c6ccf0c0-c318-4714-9168-8587c849e9dd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:53:12.565837 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] The token cache is being invalidate because project a721304960684fda928acfd30cb4434a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:53:12.566056 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:53:12.566715 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c6ccf0c0-c318-4714-9168-8587c849e9dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:53:12.568307 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1438/2851] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:53:12 2022] DELETE /identity/v3/projects/a721304960684fda928acfd30cb4434a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:53:12.573535 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa17ed-1336-4cee-9fa9-7385c165fb9c None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:12.574004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa17ed-1336-4cee-9fa9-7385c165fb9c None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:12.574262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa17ed-1336-4cee-9fa9-7385c165fb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:12.594378 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-9caa17ed-1336-4cee-9fa9-7385c165fb9c None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:53:12.595625 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-9caa17ed-1336-4cee-9fa9-7385c165fb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:12.616018 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:53:12.661526 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1414/2852] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:53:12.670012 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5aa6-38da-40a5-bf8d-f2ead514868a None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:12.670446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5aa6-38da-40a5-bf8d-f2ead514868a None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:12.670909 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5aa6-38da-40a5-bf8d-f2ead514868a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:12.699800 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-c82e5aa6-38da-40a5-bf8d-f2ead514868a None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:53:12.701008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-c82e5aa6-38da-40a5-bf8d-f2ead514868a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:12.710408 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1439/2853] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:53:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:53:12.716899 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa17ed-1336-4cee-9fa9-7385c165fb9c None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:12.731424 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa17ed-1336-4cee-9fa9-7385c165fb9c None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:12.736432 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233c5b9-632e-4e9c-bd1c-e6af352e8615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:12.737104 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233c5b9-632e-4e9c-bd1c-e6af352e8615 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:12.737383 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233c5b9-632e-4e9c-bd1c-e6af352e8615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:12.737656 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233c5b9-632e-4e9c-bd1c-e6af352e8615 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:12.738397 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233c5b9-632e-4e9c-bd1c-e6af352e8615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:53:12.738723 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233c5b9-632e-4e9c-bd1c-e6af352e8615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:12.740369 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233c5b9-632e-4e9c-bd1c-e6af352e8615 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:12.752060 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1415/2854] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:53:12.756311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e5aa6-38da-40a5-bf8d-f2ead514868a None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:12.765872 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e5aa6-38da-40a5-bf8d-f2ead514868a None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:12.770951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8623fd-3130-46e2-bcf7-1db12c7ad236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:12.771596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8623fd-3130-46e2-bcf7-1db12c7ad236 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:12.771875 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8623fd-3130-46e2-bcf7-1db12c7ad236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:12.772146 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8623fd-3130-46e2-bcf7-1db12c7ad236 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:12.772832 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8623fd-3130-46e2-bcf7-1db12c7ad236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:12.774730 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8623fd-3130-46e2-bcf7-1db12c7ad236 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:12.802235 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1440/2855] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:53:12 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:53:12.806465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233c5b9-632e-4e9c-bd1c-e6af352e8615 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:12.817081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233c5b9-632e-4e9c-bd1c-e6af352e8615 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:12.882576 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9f276-82ba-4c1a-a094-7e6c67a39692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:12.883484 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9f276-82ba-4c1a-a094-7e6c67a39692 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:12.883893 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9f276-82ba-4c1a-a094-7e6c67a39692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:12.884180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9f276-82ba-4c1a-a094-7e6c67a39692 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:12.884949 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9f276-82ba-4c1a-a094-7e6c67a39692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:12.887057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9f276-82ba-4c1a-a094-7e6c67a39692 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:12.909647 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1416/2856] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:53:12 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:53:12.914018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8623fd-3130-46e2-bcf7-1db12c7ad236 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:12.924855 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8623fd-3130-46e2-bcf7-1db12c7ad236 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:12.929922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28655a-479d-457d-8d1a-8e0e7b127473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:12.930595 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28655a-479d-457d-8d1a-8e0e7b127473 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:12.930910 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28655a-479d-457d-8d1a-8e0e7b127473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:12.931191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28655a-479d-457d-8d1a-8e0e7b127473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:12.931926 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28655a-479d-457d-8d1a-8e0e7b127473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:12.933640 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28655a-479d-457d-8d1a-8e0e7b127473 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:12.945045 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1441/2857] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:53:12.949931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9f276-82ba-4c1a-a094-7e6c67a39692 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:12.965881 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9f276-82ba-4c1a-a094-7e6c67a39692 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:12.971491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15433701-6121-4038-8167-0bd550c742c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:12.972162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15433701-6121-4038-8167-0bd550c742c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:12.972445 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15433701-6121-4038-8167-0bd550c742c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:12.972721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15433701-6121-4038-8167-0bd550c742c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:12.973428 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15433701-6121-4038-8167-0bd550c742c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:12.975099 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15433701-6121-4038-8167-0bd550c742c4 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:12.985884 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1417/2858] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:53:12.990420 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28655a-479d-457d-8d1a-8e0e7b127473 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:13.001015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28655a-479d-457d-8d1a-8e0e7b127473 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:13.006255 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d96a31-6424-4422-8f71-f9920197b8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:13.007188 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d96a31-6424-4422-8f71-f9920197b8eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:13.007496 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d96a31-6424-4422-8f71-f9920197b8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:13.007766 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d96a31-6424-4422-8f71-f9920197b8eb admin admin] PATH_INFO: `/v3/projects/37185eb6b2dd4375a6909aaefe1cc7f5/users/3ed513efcdf04cc48abcb078e857c025/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:13.022797 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d96a31-6424-4422-8f71-f9920197b8eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=37185eb6b2dd4375a6909aaefe1cc7f5, user_id=3ed513efcdf04cc48abcb078e857c025, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:13.025434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d96a31-6424-4422-8f71-f9920197b8eb admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:13.047546 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1442/2859] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:53:12 2022] PUT /identity/v3/projects/37185eb6b2dd4375a6909aaefe1cc7f5/users/3ed513efcdf04cc48abcb078e857c025/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:53:13.055729 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc84d-53a9-4465-be8f-7591e768d5fb None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:13.056203 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc84d-53a9-4465-be8f-7591e768d5fb None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:13.056650 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfc84d-53a9-4465-be8f-7591e768d5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:13.076236 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-3acfc84d-53a9-4465-be8f-7591e768d5fb None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:53:13.077364 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-3acfc84d-53a9-4465-be8f-7591e768d5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:13.141149 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1418/2860] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:53:13.152726 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d96a31-6424-4422-8f71-f9920197b8eb admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:13.154760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-a4d96a31-6424-4422-8f71-f9920197b8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:13.177268 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:53:13.216474 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d96a31-6424-4422-8f71-f9920197b8eb admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:13.221138 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150846dd-4da8-4783-971d-c7ebc41953ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:13.221816 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150846dd-4da8-4783-971d-c7ebc41953ea service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:13.222077 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150846dd-4da8-4783-971d-c7ebc41953ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:13.222350 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150846dd-4da8-4783-971d-c7ebc41953ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:13.232735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150846dd-4da8-4783-971d-c7ebc41953ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:13.234202 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150846dd-4da8-4783-971d-c7ebc41953ea service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:13.244503 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1443/2861] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:53:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:53:13.891076 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5970258-d199-4aa3-92e4-cf1c35f9f8b9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:13.891648 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5970258-d199-4aa3-92e4-cf1c35f9f8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:13.891773 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5970258-d199-4aa3-92e4-cf1c35f9f8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:13.918306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-f5970258-d199-4aa3-92e4-cf1c35f9f8b9 None None] MFA Rules not processed for user `3ed513efcdf04cc48abcb078e857c025`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:53:13.919553 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-f5970258-d199-4aa3-92e4-cf1c35f9f8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:13.937579 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:53:13.992093 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1419/2862] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:53:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:53:14.003076 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150846dd-4da8-4783-971d-c7ebc41953ea service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:14.004491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-150846dd-4da8-4783-971d-c7ebc41953ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:14.064506 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150846dd-4da8-4783-971d-c7ebc41953ea service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:14.068477 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf47857-2958-4edc-90ce-2c521456eb25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:14.069194 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47857-2958-4edc-90ce-2c521456eb25 service cinder] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:14.069491 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47857-2958-4edc-90ce-2c521456eb25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:14.069759 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47857-2958-4edc-90ce-2c521456eb25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:14.078863 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf47857-2958-4edc-90ce-2c521456eb25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:14.080294 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf47857-2958-4edc-90ce-2c521456eb25 service cinder] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:14.089524 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1444/2863] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:53:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:53:15.448285 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd387225-c998-4a4a-bc26-6cb1850d6814 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:15.448859 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd387225-c998-4a4a-bc26-6cb1850d6814 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:15.449460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd387225-c998-4a4a-bc26-6cb1850d6814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:15.471164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-bd387225-c998-4a4a-bc26-6cb1850d6814 None None] MFA Rules not processed for user `3ed513efcdf04cc48abcb078e857c025`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:53:15.472389 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bd387225-c998-4a4a-bc26-6cb1850d6814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:15.481247 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1420/2864] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:53:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:53:15.731610 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf47857-2958-4edc-90ce-2c521456eb25 service cinder] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:15.733199 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-cdf47857-2958-4edc-90ce-2c521456eb25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:15.747292 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf47857-2958-4edc-90ce-2c521456eb25 service cinder] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:15.751456 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de195b-c73e-450d-9559-df9a70f39192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:15.752097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de195b-c73e-450d-9559-df9a70f39192 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:15.752482 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de195b-c73e-450d-9559-df9a70f39192 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:15.752793 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de195b-c73e-450d-9559-df9a70f39192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:15.754630 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-70de195b-c73e-450d-9559-df9a70f39192 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:15.764332 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de195b-c73e-450d-9559-df9a70f39192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:15.765761 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de195b-c73e-450d-9559-df9a70f39192 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:15.773923 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1445/2865] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:53:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:53:16.107069 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd387225-c998-4a4a-bc26-6cb1850d6814 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:16.109014 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bd387225-c998-4a4a-bc26-6cb1850d6814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:16.187771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd387225-c998-4a4a-bc26-6cb1850d6814 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:16.193053 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb7baa-067f-45bf-8fcd-9465a91658c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:16.193713 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb7baa-067f-45bf-8fcd-9465a91658c5 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:16.193997 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb7baa-067f-45bf-8fcd-9465a91658c5 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:16.194275 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb7baa-067f-45bf-8fcd-9465a91658c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:16.196187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-55bb7baa-067f-45bf-8fcd-9465a91658c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:16.208051 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb7baa-067f-45bf-8fcd-9465a91658c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:16.209573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb7baa-067f-45bf-8fcd-9465a91658c5 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:16.220362 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1421/2866] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:53:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:53:25.636515 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de195b-c73e-450d-9559-df9a70f39192 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:25.637963 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-70de195b-c73e-450d-9559-df9a70f39192 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:25.715124 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de195b-c73e-450d-9559-df9a70f39192 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:25.720764 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc8c7a-b195-43b9-8397-a91216c67318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:25.721410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8c7a-b195-43b9-8397-a91216c67318 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:25.721685 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8c7a-b195-43b9-8397-a91216c67318 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:25.721960 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc8c7a-b195-43b9-8397-a91216c67318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:25.724047 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-16bc8c7a-b195-43b9-8397-a91216c67318 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:25.736310 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc8c7a-b195-43b9-8397-a91216c67318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:25.738018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc8c7a-b195-43b9-8397-a91216c67318 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:25.748721 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1446/2867] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:53:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:53:26.215921 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=54 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:53:32.067703 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ce327-4f5e-4ad7-bf0a-19425e1331d6 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:32.068341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ce327-4f5e-4ad7-bf0a-19425e1331d6 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:32.068341 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ce327-4f5e-4ad7-bf0a-19425e1331d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:32.089585 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-b93ce327-4f5e-4ad7-bf0a-19425e1331d6 None None] MFA Rules not processed for user `3ed513efcdf04cc48abcb078e857c025`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:53:32.090819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-b93ce327-4f5e-4ad7-bf0a-19425e1331d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:32.098144 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1422/2868] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:53:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:53:32.332002 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc8c7a-b195-43b9-8397-a91216c67318 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:32.342578 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc8c7a-b195-43b9-8397-a91216c67318 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:32.347175 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4e2d6-3e52-4767-ba6e-be791cf89bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:32.347823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4e2d6-3e52-4767-ba6e-be791cf89bef service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:32.348051 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4e2d6-3e52-4767-ba6e-be791cf89bef service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:32.348349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4e2d6-3e52-4767-ba6e-be791cf89bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:32.350428 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-69e4e2d6-3e52-4767-ba6e-be791cf89bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:32.363050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4e2d6-3e52-4767-ba6e-be791cf89bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:32.364502 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4e2d6-3e52-4767-ba6e-be791cf89bef service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:32.372773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1447/2869] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:53:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:53:42.082813 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:53:42.369396 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:53:57.815847 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77da52-b28c-4917-b34a-43a34a00f38f None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:57.816460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77da52-b28c-4917-b34a-43a34a00f38f None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:57.816460 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77da52-b28c-4917-b34a-43a34a00f38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:57.877453 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-6a77da52-b28c-4917-b34a-43a34a00f38f None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:53:57.878677 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6a77da52-b28c-4917-b34a-43a34a00f38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:57.946856 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1423/2870] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:53:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:53:58.002211 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4e2d6-3e52-4767-ba6e-be791cf89bef service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:53:58.003707 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-69e4e2d6-3e52-4767-ba6e-be791cf89bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:53:58.098235 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4e2d6-3e52-4767-ba6e-be791cf89bef service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:53:58.104109 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded0acf-b5c0-4cf5-a93a-31d2e422b639 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:53:58.104752 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded0acf-b5c0-4cf5-a93a-31d2e422b639 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:53:58.105032 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded0acf-b5c0-4cf5-a93a-31d2e422b639 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:53:58.105309 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded0acf-b5c0-4cf5-a93a-31d2e422b639 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:53:58.115434 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded0acf-b5c0-4cf5-a93a-31d2e422b639 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:53:58.117063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded0acf-b5c0-4cf5-a93a-31d2e422b639 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:53:58.125288 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1448/2871] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:53:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:54:07.943715 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=44 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:54:08.122142 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=41 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:54:12.279911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602cc6a-de05-4589-8eab-28990a3452b1 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:12.280421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602cc6a-de05-4589-8eab-28990a3452b1 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:12.280421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602cc6a-de05-4589-8eab-28990a3452b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:12.298744 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-5602cc6a-de05-4589-8eab-28990a3452b1 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:54:12.299934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5602cc6a-de05-4589-8eab-28990a3452b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:12.307988 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1424/2872] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jun 29 08:54:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:54:12.320903 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded0acf-b5c0-4cf5-a93a-31d2e422b639 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:12.336001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded0acf-b5c0-4cf5-a93a-31d2e422b639 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:12.342840 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b81621-08db-47ab-93e8-95ef0d31aae2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:12.343464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b81621-08db-47ab-93e8-95ef0d31aae2 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:12.343760 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b81621-08db-47ab-93e8-95ef0d31aae2 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:12.344033 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b81621-08db-47ab-93e8-95ef0d31aae2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:12.352738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b81621-08db-47ab-93e8-95ef0d31aae2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:12.354323 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b81621-08db-47ab-93e8-95ef0d31aae2 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:12.362464 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1449/2873] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:54:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:54:14.268984 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602cc6a-de05-4589-8eab-28990a3452b1 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:14.270467 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5602cc6a-de05-4589-8eab-28990a3452b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:14.285644 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602cc6a-de05-4589-8eab-28990a3452b1 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:14.291234 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6707d6-f378-4226-9164-4ab5ea8a1872 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:14.291942 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6707d6-f378-4226-9164-4ab5ea8a1872 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:14.292224 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6707d6-f378-4226-9164-4ab5ea8a1872 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:14.292502 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6707d6-f378-4226-9164-4ab5ea8a1872 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:14.294248 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-ee6707d6-f378-4226-9164-4ab5ea8a1872 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:14.307296 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6707d6-f378-4226-9164-4ab5ea8a1872 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:14.308979 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6707d6-f378-4226-9164-4ab5ea8a1872 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:14.323882 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b81621-08db-47ab-93e8-95ef0d31aae2 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:14.336511 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b81621-08db-47ab-93e8-95ef0d31aae2 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:14.342050 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495037c-e151-49df-a92e-758c7d705179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:14.342279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1425/2874] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:54:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:54:14.343176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495037c-e151-49df-a92e-758c7d705179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:14.343525 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495037c-e151-49df-a92e-758c7d705179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:14.343873 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495037c-e151-49df-a92e-758c7d705179 admin admin] PATH_INFO: `/v3/users/3ed513efcdf04cc48abcb078e857c025` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:14.345176 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495037c-e151-49df-a92e-758c7d705179 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ed513efcdf04cc48abcb078e857c025)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:14.347137 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495037c-e151-49df-a92e-758c7d705179 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:14.397327 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7495037c-e151-49df-a92e-758c7d705179 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ed513efcdf04cc48abcb078e857c025'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:14.397971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7495037c-e151-49df-a92e-758c7d705179 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ed513efcdf04cc48abcb078e857c025'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:14.409789 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7495037c-e151-49df-a92e-758c7d705179 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ed513efcdf04cc48abcb078e857c025'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:14.410470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7495037c-e151-49df-a92e-758c7d705179 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ed513efcdf04cc48abcb078e857c025'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:14.418982 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-7495037c-e151-49df-a92e-758c7d705179 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ed513efcdf04cc48abcb078e857c025'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:14.431738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1450/2875] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:54:14 2022] DELETE /identity/v3/users/3ed513efcdf04cc48abcb078e857c025 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:54:14.622747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6707d6-f378-4226-9164-4ab5ea8a1872 service placement] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:14.633669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6707d6-f378-4226-9164-4ab5ea8a1872 service placement] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:14.691034 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:14.691906 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:14.692184 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:14.692455 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] PATH_INFO: `/v3/projects/37185eb6b2dd4375a6909aaefe1cc7f5` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:14.693661 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] RBAC: Authorizing `identity:delete_project(project_id=37185eb6b2dd4375a6909aaefe1cc7f5)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:14.695611 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:14.743645 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37185eb6b2dd4375a6909aaefe1cc7f5', 'request_id': 'req-fc7e5794-76be-4607-bee0-7fe6768d519f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:14.749946 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37185eb6b2dd4375a6909aaefe1cc7f5', 'request_id': 'req-fc7e5794-76be-4607-bee0-7fe6768d519f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:14.756672 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37185eb6b2dd4375a6909aaefe1cc7f5', 'request_id': 'req-fc7e5794-76be-4607-bee0-7fe6768d519f'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:14.776386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] The token cache is being invalidate because project 37185eb6b2dd4375a6909aaefe1cc7f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80130) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:54:14.776816 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:14.777976 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-fc7e5794-76be-4607-bee0-7fe6768d519f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:14.779721 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1426/2876] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:54:14 2022] DELETE /identity/v3/projects/37185eb6b2dd4375a6909aaefe1cc7f5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:54:14.786378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea97b7-7c79-4d98-8417-03c0e4ef8f4b None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:14.786957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea97b7-7c79-4d98-8417-03c0e4ef8f4b None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:14.787316 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea97b7-7c79-4d98-8417-03c0e4ef8f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:14.806711 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-f5ea97b7-7c79-4d98-8417-03c0e4ef8f4b None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:54:14.808038 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-f5ea97b7-7c79-4d98-8417-03c0e4ef8f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:14.868488 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1451/2877] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:54:14.876710 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ae868-6c30-4e7e-ba9f-fe814fff9544 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:14.876996 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ae868-6c30-4e7e-ba9f-fe814fff9544 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:14.877273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ae868-6c30-4e7e-ba9f-fe814fff9544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:14.896594 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-472ae868-6c30-4e7e-ba9f-fe814fff9544 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:54:14.897751 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-472ae868-6c30-4e7e-ba9f-fe814fff9544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:14.906497 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1427/2878] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:54:14.910878 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea97b7-7c79-4d98-8417-03c0e4ef8f4b None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:14.920687 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea97b7-7c79-4d98-8417-03c0e4ef8f4b None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:14.925555 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6823268-3400-4f36-afd9-94b5a35ae686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:14.926212 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6823268-3400-4f36-afd9-94b5a35ae686 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:14.926486 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6823268-3400-4f36-afd9-94b5a35ae686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:14.926791 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6823268-3400-4f36-afd9-94b5a35ae686 admin admin] PATH_INFO: `/v3/domains` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:14.927530 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6823268-3400-4f36-afd9-94b5a35ae686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80131) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:54:14.927830 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6823268-3400-4f36-afd9-94b5a35ae686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:14.929430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6823268-3400-4f36-afd9-94b5a35ae686 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:14.941144 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1452/2879] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:54:14.946059 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ae868-6c30-4e7e-ba9f-fe814fff9544 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:14.960894 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ae868-6c30-4e7e-ba9f-fe814fff9544 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:14.968077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30880a-48a1-49e5-8476-dbb42b38288f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:14.968735 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30880a-48a1-49e5-8476-dbb42b38288f admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:14.969015 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30880a-48a1-49e5-8476-dbb42b38288f admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:14.969279 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30880a-48a1-49e5-8476-dbb42b38288f admin admin] PATH_INFO: `/v3/projects` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:14.969971 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30880a-48a1-49e5-8476-dbb42b38288f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:14.971900 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30880a-48a1-49e5-8476-dbb42b38288f admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:15.011685 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1428/2880] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:54:14 2022] POST /identity/v3/projects => generated 363 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:54:15.012904 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6823268-3400-4f36-afd9-94b5a35ae686 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:15.023018 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6823268-3400-4f36-afd9-94b5a35ae686 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:15.066576 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:54:15.081916 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08206e3-b5b6-4d9e-833f-0c965233a3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:15.082586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08206e3-b5b6-4d9e-833f-0c965233a3d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:15.082906 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08206e3-b5b6-4d9e-833f-0c965233a3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:15.083186 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08206e3-b5b6-4d9e-833f-0c965233a3d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:15.084063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08206e3-b5b6-4d9e-833f-0c965233a3d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:15.085865 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08206e3-b5b6-4d9e-833f-0c965233a3d9 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:15.113349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1453/2881] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:54:15 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 08:54:15.119114 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30880a-48a1-49e5-8476-dbb42b38288f admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:15.130639 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30880a-48a1-49e5-8476-dbb42b38288f admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:15.136517 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae7f9c-a061-406f-80f8-1cb78bbc3cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:15.137196 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae7f9c-a061-406f-80f8-1cb78bbc3cbc admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:15.137481 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae7f9c-a061-406f-80f8-1cb78bbc3cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:15.137764 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae7f9c-a061-406f-80f8-1cb78bbc3cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:15.138482 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae7f9c-a061-406f-80f8-1cb78bbc3cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:15.140175 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae7f9c-a061-406f-80f8-1cb78bbc3cbc admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:15.151551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1429/2882] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:54:15.155826 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08206e3-b5b6-4d9e-833f-0c965233a3d9 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:15.169191 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08206e3-b5b6-4d9e-833f-0c965233a3d9 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:15.174922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09317414-6265-41d9-9ea6-871240c62ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:15.175639 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09317414-6265-41d9-9ea6-871240c62ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:15.175940 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09317414-6265-41d9-9ea6-871240c62ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:15.176206 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09317414-6265-41d9-9ea6-871240c62ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:15.177022 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09317414-6265-41d9-9ea6-871240c62ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:15.179461 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09317414-6265-41d9-9ea6-871240c62ce0 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:15.189972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1454/2883] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:54:15.194910 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae7f9c-a061-406f-80f8-1cb78bbc3cbc admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:15.205173 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae7f9c-a061-406f-80f8-1cb78bbc3cbc admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:15.211049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3e0a3-6bd1-45b9-8d93-bf4589b7b226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:15.211943 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3e0a3-6bd1-45b9-8d93-bf4589b7b226 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:15.212237 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3e0a3-6bd1-45b9-8d93-bf4589b7b226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:15.212860 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3e0a3-6bd1-45b9-8d93-bf4589b7b226 admin admin] PATH_INFO: `/v3/projects/0a42730c4f664742968feaac754c8873/users/40e519db9c1f44bc9953eebebd025429/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:15.229340 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3e0a3-6bd1-45b9-8d93-bf4589b7b226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a42730c4f664742968feaac754c8873, user_id=40e519db9c1f44bc9953eebebd025429, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:15.233765 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3e0a3-6bd1-45b9-8d93-bf4589b7b226 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:15.251669 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1430/2884] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:54:15 2022] PUT /identity/v3/projects/0a42730c4f664742968feaac754c8873/users/40e519db9c1f44bc9953eebebd025429/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:54:15.264058 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8210d-8d86-4308-be8f-7c5dd571bfa5 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:15.264352 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8210d-8d86-4308-be8f-7c5dd571bfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:15.264633 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8210d-8d86-4308-be8f-7c5dd571bfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:15.291600 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-4fd8210d-8d86-4308-be8f-7c5dd571bfa5 None None] MFA Rules not processed for user `40e519db9c1f44bc9953eebebd025429`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:54:15.292805 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4fd8210d-8d86-4308-be8f-7c5dd571bfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:15.375034 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1455/2885] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:54:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:54:15.389754 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3e0a3-6bd1-45b9-8d93-bf4589b7b226 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:15.392452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-bce3e0a3-6bd1-45b9-8d93-bf4589b7b226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:15.404702 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:54:15.455514 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3e0a3-6bd1-45b9-8d93-bf4589b7b226 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:15.459640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7200c4-197f-470b-b5a0-7c82243ec7b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b60ca33edf5a47fdab6e39cc98638c6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:15.460307 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7200c4-197f-470b-b5a0-7c82243ec7b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:15.460593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7200c4-197f-470b-b5a0-7c82243ec7b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:15.460873 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7200c4-197f-470b-b5a0-7c82243ec7b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:15.470109 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7200c4-197f-470b-b5a0-7c82243ec7b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:15.471622 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7200c4-197f-470b-b5a0-7c82243ec7b6 service cinder] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:15.481063 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1431/2886] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:54:15 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:54:25.371992 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:54:25.476948 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=23 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:54:27.081651 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8210d-8d86-4308-be8f-7c5dd571bfa5 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:27.083972 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-4fd8210d-8d86-4308-be8f-7c5dd571bfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:27.162365 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8210d-8d86-4308-be8f-7c5dd571bfa5 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:27.168612 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80577ff7-6aa1-41a9-9854-3bc1b4f06ff9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:27.170538 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80577ff7-6aa1-41a9-9854-3bc1b4f06ff9 service placement] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:27.170858 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80577ff7-6aa1-41a9-9854-3bc1b4f06ff9 service placement] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:27.171141 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80577ff7-6aa1-41a9-9854-3bc1b4f06ff9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:27.173231 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-80577ff7-6aa1-41a9-9854-3bc1b4f06ff9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:27.186442 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80577ff7-6aa1-41a9-9854-3bc1b4f06ff9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:27.188093 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80577ff7-6aa1-41a9-9854-3bc1b4f06ff9 service placement] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:27.198159 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1456/2887] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:54:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:54:37.194196 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=32 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:54:57.134887 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7200c4-197f-470b-b5a0-7c82243ec7b6 service cinder] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:57.209772 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7200c4-197f-470b-b5a0-7c82243ec7b6 service cinder] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:57.216665 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:57.217537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:57.217793 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:57.218080 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] PATH_INFO: `/v3/users/40e519db9c1f44bc9953eebebd025429` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:57.219330 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e519db9c1f44bc9953eebebd025429)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:57.221262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:57.271211 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e519db9c1f44bc9953eebebd025429'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:57.271771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e519db9c1f44bc9953eebebd025429'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:57.283573 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e519db9c1f44bc9953eebebd025429'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:57.284201 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e519db9c1f44bc9953eebebd025429'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:57.293081 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e519db9c1f44bc9953eebebd025429'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:57.305291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1432/2888] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:54:57 2022] DELETE /identity/v3/users/40e519db9c1f44bc9953eebebd025429 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:54:57.310430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b411afe-0a43-4673-bb6b-fa2069909699 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:57.311010 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b411afe-0a43-4673-bb6b-fa2069909699 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:57.311126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b411afe-0a43-4673-bb6b-fa2069909699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:57.329835 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9b411afe-0a43-4673-bb6b-fa2069909699 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:54:57.331104 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9b411afe-0a43-4673-bb6b-fa2069909699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:57.392867 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1457/2889] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:54:57.403416 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:57.404916 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:57.467732 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc6451-3777-44d4-bdaa-912291db8133 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:57.474273 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a0ed1-43ba-4e82-a65c-0b3a716f7b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:57.475327 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a0ed1-43ba-4e82-a65c-0b3a716f7b92 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:57.475681 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a0ed1-43ba-4e82-a65c-0b3a716f7b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:57.476057 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a0ed1-43ba-4e82-a65c-0b3a716f7b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:57.488087 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a0ed1-43ba-4e82-a65c-0b3a716f7b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:57.489547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a0ed1-43ba-4e82-a65c-0b3a716f7b92 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:57.500077 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1433/2890] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:54:57.790125 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b411afe-0a43-4673-bb6b-fa2069909699 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:57.791636 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9b411afe-0a43-4673-bb6b-fa2069909699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:57.803611 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b411afe-0a43-4673-bb6b-fa2069909699 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:57.808513 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:57.813220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:57.813220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:57.813220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] PATH_INFO: `/v3/projects/0a42730c4f664742968feaac754c8873` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:57.813220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a42730c4f664742968feaac754c8873)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:57.813220 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:57.864855 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a42730c4f664742968feaac754c8873', 'request_id': 'req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:57.871017 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a42730c4f664742968feaac754c8873', 'request_id': 'req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:57.876815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a42730c4f664742968feaac754c8873', 'request_id': 'req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:57.892824 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] The token cache is being invalidate because project 0a42730c4f664742968feaac754c8873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:54:57.893116 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:57.893773 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-c2a6ca12-c4a2-416c-86a9-9e11a927f3bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:54:57.895389 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1458/2891] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:54:57 2022] DELETE /identity/v3/projects/0a42730c4f664742968feaac754c8873 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:54:57.900431 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdbb59-8805-45c7-90f6-92b4ea8e1491 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:57.900832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdbb59-8805-45c7-90f6-92b4ea8e1491 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:57.901164 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdbb59-8805-45c7-90f6-92b4ea8e1491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:57.919409 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-99bdbb59-8805-45c7-90f6-92b4ea8e1491 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:54:57.920624 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-99bdbb59-8805-45c7-90f6-92b4ea8e1491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:57.938026 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:54:57.990797 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1434/2892] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:54:58.003215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5c47d-a7be-4f3d-9f99-44ccd4264dcd None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:58.003603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5c47d-a7be-4f3d-9f99-44ccd4264dcd None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:58.003977 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5c47d-a7be-4f3d-9f99-44ccd4264dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:58.025139 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-9ae5c47d-a7be-4f3d-9f99-44ccd4264dcd None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:54:58.026390 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-9ae5c47d-a7be-4f3d-9f99-44ccd4264dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:58.035628 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1459/2893] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:54:58.039819 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bdbb59-8805-45c7-90f6-92b4ea8e1491 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:58.050958 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bdbb59-8805-45c7-90f6-92b4ea8e1491 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:58.057115 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31006d05-dd7e-465e-8a7f-77f07f9264d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:58.057780 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31006d05-dd7e-465e-8a7f-77f07f9264d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:58.058066 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31006d05-dd7e-465e-8a7f-77f07f9264d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:58.058344 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31006d05-dd7e-465e-8a7f-77f07f9264d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:58.059123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31006d05-dd7e-465e-8a7f-77f07f9264d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80130) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 08:54:58.059435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31006d05-dd7e-465e-8a7f-77f07f9264d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:58.061046 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31006d05-dd7e-465e-8a7f-77f07f9264d3 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:58.073855 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1435/2894] 10.222.0.27 () {58 vars in 1152 bytes} [Wed Jun 29 08:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 08:54:58.078378 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5c47d-a7be-4f3d-9f99-44ccd4264dcd None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:58.088122 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5c47d-a7be-4f3d-9f99-44ccd4264dcd None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:58.093895 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef7cb7-0b83-45d9-a88f-5ccf3d9e3a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:58.094558 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef7cb7-0b83-45d9-a88f-5ccf3d9e3a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:58.094890 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef7cb7-0b83-45d9-a88f-5ccf3d9e3a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:58.095165 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef7cb7-0b83-45d9-a88f-5ccf3d9e3a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:58.095866 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef7cb7-0b83-45d9-a88f-5ccf3d9e3a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:58.097815 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef7cb7-0b83-45d9-a88f-5ccf3d9e3a04 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:58.127848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1460/2895] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:54:58 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:54:58.132071 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31006d05-dd7e-465e-8a7f-77f07f9264d3 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:58.145162 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31006d05-dd7e-465e-8a7f-77f07f9264d3 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:58.215876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d02e0e-f9e1-4c75-b1c2-f41952b517e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:58.216558 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d02e0e-f9e1-4c75-b1c2-f41952b517e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:58.216842 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d02e0e-f9e1-4c75-b1c2-f41952b517e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:58.217123 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d02e0e-f9e1-4c75-b1c2-f41952b517e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:58.217824 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d02e0e-f9e1-4c75-b1c2-f41952b517e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:58.219812 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d02e0e-f9e1-4c75-b1c2-f41952b517e5 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:58.241736 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1436/2896] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:54:58 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:54:58.245922 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef7cb7-0b83-45d9-a88f-5ccf3d9e3a04 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:58.256409 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef7cb7-0b83-45d9-a88f-5ccf3d9e3a04 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:58.262237 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be363359-6f30-4b41-9e2a-aae70b836656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:58.262941 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be363359-6f30-4b41-9e2a-aae70b836656 admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:58.263224 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be363359-6f30-4b41-9e2a-aae70b836656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:58.263494 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be363359-6f30-4b41-9e2a-aae70b836656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:58.264226 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be363359-6f30-4b41-9e2a-aae70b836656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:58.265896 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be363359-6f30-4b41-9e2a-aae70b836656 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:58.277097 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1461/2897] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:54:58.281448 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d02e0e-f9e1-4c75-b1c2-f41952b517e5 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:58.298005 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d02e0e-f9e1-4c75-b1c2-f41952b517e5 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:58.307401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9c1e7-9095-47d7-8a5c-c689ce15d356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:58.308097 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9c1e7-9095-47d7-8a5c-c689ce15d356 admin admin] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:58.308383 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9c1e7-9095-47d7-8a5c-c689ce15d356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:58.308659 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9c1e7-9095-47d7-8a5c-c689ce15d356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:58.309402 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9c1e7-9095-47d7-8a5c-c689ce15d356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:58.311032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9c1e7-9095-47d7-8a5c-c689ce15d356 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:58.321435 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1437/2898] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:54:58.327551 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be363359-6f30-4b41-9e2a-aae70b836656 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:58.338229 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be363359-6f30-4b41-9e2a-aae70b836656 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:58.343135 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0129fb8-6043-4c9c-bb87-cda4f7ec194c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:58.344024 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0129fb8-6043-4c9c-bb87-cda4f7ec194c admin admin] REQUEST_METHOD: `PUT` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:58.344311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0129fb8-6043-4c9c-bb87-cda4f7ec194c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:58.344586 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0129fb8-6043-4c9c-bb87-cda4f7ec194c admin admin] PATH_INFO: `/v3/projects/50a457f985c4496ca41152bfa29ae9e3/users/340cc4d0b4c543fab577749f947598b1/roles/b0d377da1a7d4260be1f7ae0715fda91` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:58.360410 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0129fb8-6043-4c9c-bb87-cda4f7ec194c admin admin] RBAC: Authorizing `identity:create_grant(project_id=50a457f985c4496ca41152bfa29ae9e3, user_id=340cc4d0b4c543fab577749f947598b1, role_id=b0d377da1a7d4260be1f7ae0715fda91)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:58.363114 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0129fb8-6043-4c9c-bb87-cda4f7ec194c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:58.381655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1462/2899] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:54:58 2022] PUT /identity/v3/projects/50a457f985c4496ca41152bfa29ae9e3/users/340cc4d0b4c543fab577749f947598b1/roles/b0d377da1a7d4260be1f7ae0715fda91 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:54:58.386964 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4954134-ea2e-46cf-a3e9-8e3098edf8f3 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:58.387259 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4954134-ea2e-46cf-a3e9-8e3098edf8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:58.387534 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4954134-ea2e-46cf-a3e9-8e3098edf8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:58.405276 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-a4954134-ea2e-46cf-a3e9-8e3098edf8f3 None None] MFA Rules not processed for user `c145cd66ea9b4d3a85c89c24f585acff`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:54:58.406499 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-a4954134-ea2e-46cf-a3e9-8e3098edf8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:58.470191 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1438/2900] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:54:58.480162 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0129fb8-6043-4c9c-bb87-cda4f7ec194c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:54:58.482156 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-e0129fb8-6043-4c9c-bb87-cda4f7ec194c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:54:58.505300 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:54:58.543244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0129fb8-6043-4c9c-bb87-cda4f7ec194c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:54:58.547421 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19a42e-a663-49e4-8ab5-edcf2d744896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:54:58.548063 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19a42e-a663-49e4-8ab5-edcf2d744896 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:54:58.548341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19a42e-a663-49e4-8ab5-edcf2d744896 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:54:58.548616 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b19a42e-a663-49e4-8ab5-edcf2d744896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:54:58.558531 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19a42e-a663-49e4-8ab5-edcf2d744896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:54:58.559993 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b19a42e-a663-49e4-8ab5-edcf2d744896 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:54:58.569407 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1463/2901] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:54:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:55:02.370022 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64dfd2-571a-4d4a-96cf-1c36ee4f3fb9 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:02.370978 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64dfd2-571a-4d4a-96cf-1c36ee4f3fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:02.371262 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64dfd2-571a-4d4a-96cf-1c36ee4f3fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:02.401209 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-4b64dfd2-571a-4d4a-96cf-1c36ee4f3fb9 None None] MFA Rules not processed for user `340cc4d0b4c543fab577749f947598b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:55:02.402421 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-4b64dfd2-571a-4d4a-96cf-1c36ee4f3fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:55:02.421882 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:55:02.475576 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1439/2902] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:55:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:55:02.480174 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19a42e-a663-49e4-8ab5-edcf2d744896 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:55:02.490205 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b19a42e-a663-49e4-8ab5-edcf2d744896 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:55:02.495214 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6705-6191-4b66-86e5-fe89c80e481a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:55:02.495971 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6705-6191-4b66-86e5-fe89c80e481a admin admin] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:02.496254 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6705-6191-4b66-86e5-fe89c80e481a admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:02.496527 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec6705-6191-4b66-86e5-fe89c80e481a admin admin] PATH_INFO: `/v3/projects` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:02.497244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec6705-6191-4b66-86e5-fe89c80e481a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:55:02.499413 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec6705-6191-4b66-86e5-fe89c80e481a admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:55:02.530729 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1464/2903] 10.222.0.27 () {60 vars in 1152 bytes} [Wed Jun 29 08:55:02 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:55:02.532749 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64dfd2-571a-4d4a-96cf-1c36ee4f3fb9 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:55:02.544359 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64dfd2-571a-4d4a-96cf-1c36ee4f3fb9 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:55:02.604943 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92ff89-6de2-456f-81c2-adde9d61f130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:55:02.605632 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92ff89-6de2-456f-81c2-adde9d61f130 admin admin] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:02.605911 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92ff89-6de2-456f-81c2-adde9d61f130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:02.606180 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92ff89-6de2-456f-81c2-adde9d61f130 admin admin] PATH_INFO: `/v3/users` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:02.606925 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92ff89-6de2-456f-81c2-adde9d61f130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:55:02.608870 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92ff89-6de2-456f-81c2-adde9d61f130 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:55:02.631260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1440/2904] 10.222.0.27 () {60 vars in 1143 bytes} [Wed Jun 29 08:55:02 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 08:55:02.636001 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6705-6191-4b66-86e5-fe89c80e481a admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:55:02.646470 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec6705-6191-4b66-86e5-fe89c80e481a admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:55:02.651119 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b22a64-9487-46f8-b208-9a13b2e1ca4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:55:02.651799 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b22a64-9487-46f8-b208-9a13b2e1ca4c admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:02.652082 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b22a64-9487-46f8-b208-9a13b2e1ca4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:02.652357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b22a64-9487-46f8-b208-9a13b2e1ca4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:02.653096 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b22a64-9487-46f8-b208-9a13b2e1ca4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:55:02.654905 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b22a64-9487-46f8-b208-9a13b2e1ca4c admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:55:02.666336 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1465/2905] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 08:55:02.670784 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92ff89-6de2-456f-81c2-adde9d61f130 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:55:02.681638 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92ff89-6de2-456f-81c2-adde9d61f130 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:55:02.686950 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e8742-36cb-4d5a-a426-8343e3f0758e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:55:02.687876 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e8742-36cb-4d5a-a426-8343e3f0758e admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:02.688192 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e8742-36cb-4d5a-a426-8343e3f0758e admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:02.688471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e8742-36cb-4d5a-a426-8343e3f0758e admin admin] PATH_INFO: `/v3/projects/71a66c47fe674c598d64780f7bb261c1/users/ce44ba2e39494236878484fe3f0576f0/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:02.704538 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e8742-36cb-4d5a-a426-8343e3f0758e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a66c47fe674c598d64780f7bb261c1, user_id=ce44ba2e39494236878484fe3f0576f0, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:55:02.707228 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e8742-36cb-4d5a-a426-8343e3f0758e admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:55:02.725365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1441/2906] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:55:02 2022] PUT /identity/v3/projects/71a66c47fe674c598d64780f7bb261c1/users/ce44ba2e39494236878484fe3f0576f0/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 08:55:02.729735 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b22a64-9487-46f8-b208-9a13b2e1ca4c admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:55:02.743320 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b22a64-9487-46f8-b208-9a13b2e1ca4c admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:55:02.804505 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a46aa-c8ff-43be-af5a-8fab8f9dcccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:55:02.805183 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a46aa-c8ff-43be-af5a-8fab8f9dcccc admin admin] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:02.805469 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a46aa-c8ff-43be-af5a-8fab8f9dcccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:02.805738 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a46aa-c8ff-43be-af5a-8fab8f9dcccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:02.806471 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a46aa-c8ff-43be-af5a-8fab8f9dcccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:55:02.808174 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a46aa-c8ff-43be-af5a-8fab8f9dcccc admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:55:02.818533 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1466/2907] 10.222.0.27 () {58 vars in 1121 bytes} [Wed Jun 29 08:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 08:55:02.823212 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e8742-36cb-4d5a-a426-8343e3f0758e admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:55:02.834883 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e8742-36cb-4d5a-a426-8343e3f0758e admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:55:02.840955 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128900d1-4379-4e78-b480-c4ea4e461123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:55:02.841848 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128900d1-4379-4e78-b480-c4ea4e461123 admin admin] REQUEST_METHOD: `PUT` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:02.842132 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128900d1-4379-4e78-b480-c4ea4e461123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:02.842462 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128900d1-4379-4e78-b480-c4ea4e461123 admin admin] PATH_INFO: `/v3/projects/71a66c47fe674c598d64780f7bb261c1/users/ce44ba2e39494236878484fe3f0576f0/roles/d7286245c7bf41c9b699f41f0a2fa69e` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:02.844613 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128900d1-4379-4e78-b480-c4ea4e461123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a66c47fe674c598d64780f7bb261c1, user_id=ce44ba2e39494236878484fe3f0576f0, role_id=d7286245c7bf41c9b699f41f0a2fa69e)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:55:02.847372 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128900d1-4379-4e78-b480-c4ea4e461123 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:55:02.857835 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1442/2908] 10.222.0.27 () {60 vars in 1482 bytes} [Wed Jun 29 08:55:02 2022] PUT /identity/v3/projects/71a66c47fe674c598d64780f7bb261c1/users/ce44ba2e39494236878484fe3f0576f0/roles/d7286245c7bf41c9b699f41f0a2fa69e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:55:07.459499 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03e7c1-4ca6-49c1-b734-5fa46b4acb49 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:07.459976 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03e7c1-4ca6-49c1-b734-5fa46b4acb49 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:07.460086 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03e7c1-4ca6-49c1-b734-5fa46b4acb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:07.502339 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-de03e7c1-4ca6-49c1-b734-5fa46b4acb49 None None] MFA Rules not processed for user `ce44ba2e39494236878484fe3f0576f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:55:07.503596 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-de03e7c1-4ca6-49c1-b734-5fa46b4acb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:55:07.591931 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1467/2909] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:55:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:55:07.604229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128900d1-4379-4e78-b480-c4ea4e461123 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:55:07.606049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-128900d1-4379-4e78-b480-c4ea4e461123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:55:07.668970 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128900d1-4379-4e78-b480-c4ea4e461123 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:55:07.672551 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bba96-8863-4d8e-8cd1-e5eda41e2a4c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd09453b5db645d7b03522f91effb8d4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:55:07.673216 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bba96-8863-4d8e-8cd1-e5eda41e2a4c service glance] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:07.673497 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bba96-8863-4d8e-8cd1-e5eda41e2a4c service glance] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:07.673771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bba96-8863-4d8e-8cd1-e5eda41e2a4c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:07.700356 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:55:07.736662 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bba96-8863-4d8e-8cd1-e5eda41e2a4c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:55:07.738337 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bba96-8863-4d8e-8cd1-e5eda41e2a4c service glance] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:55:07.748186 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1443/2910] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:55:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:55:07.852536 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fea11-149a-434b-9ea3-cabc57e8fad7 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:07.852827 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fea11-149a-434b-9ea3-cabc57e8fad7 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:07.853110 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fea11-149a-434b-9ea3-cabc57e8fad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:07.873562 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-241fea11-149a-434b-9ea3-cabc57e8fad7 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:55:07.875052 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-241fea11-149a-434b-9ea3-cabc57e8fad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:55:07.881577 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:55:07.934885 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1468/2911] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:55:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:55:07.956376 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bba96-8863-4d8e-8cd1-e5eda41e2a4c service glance] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:55:07.957931 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-825bba96-8863-4d8e-8cd1-e5eda41e2a4c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:55:08.034403 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bba96-8863-4d8e-8cd1-e5eda41e2a4c service glance] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:55:08.040003 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6e113-947f-49f5-8d23-391f90a2b5e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:55:08.040663 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6e113-947f-49f5-8d23-391f90a2b5e0 service swift] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:08.040913 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6e113-947f-49f5-8d23-391f90a2b5e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:08.041283 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6e113-947f-49f5-8d23-391f90a2b5e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:08.050113 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6e113-947f-49f5-8d23-391f90a2b5e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:55:08.051747 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6e113-947f-49f5-8d23-391f90a2b5e0 service swift] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:55:08.060155 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1444/2912] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:55:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:55:17.663552 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186176bf-06b1-4893-86a4-d6d32358a1c1 None None] REQUEST_METHOD: `POST` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:17.664130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186176bf-06b1-4893-86a4-d6d32358a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:17.664130 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186176bf-06b1-4893-86a4-d6d32358a1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:17.683655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.auth.core [None req-186176bf-06b1-4893-86a4-d6d32358a1c1 None None] MFA Rules not processed for user `340cc4d0b4c543fab577749f947598b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80131) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:55:17.684862 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-186176bf-06b1-4893-86a4-d6d32358a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:55:17.699451 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1469/2913] 10.222.0.27 () {58 vars in 957 bytes} [Wed Jun 29 08:55:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:55:18.056739 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=48 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:55:19.662036 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7b1db-2332-46a6-97dd-3f0dad023742 None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:19.662610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7b1db-2332-46a6-97dd-3f0dad023742 None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:19.662610 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7b1db-2332-46a6-97dd-3f0dad023742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:19.681041 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-edc7b1db-2332-46a6-97dd-3f0dad023742 None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:55:19.682265 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-edc7b1db-2332-46a6-97dd-3f0dad023742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:55:19.690098 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1445/2914] 10.222.0.27 () {58 vars in 1009 bytes} [Wed Jun 29 08:55:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:55:19.705121 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186176bf-06b1-4893-86a4-d6d32358a1c1 None None] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:55:19.721343 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186176bf-06b1-4893-86a4-d6d32358a1c1 None None] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:55:19.726758 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e4891-3699-44ac-a18e-4d55e4f9eded service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:55:19.727455 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e4891-3699-44ac-a18e-4d55e4f9eded service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:19.727741 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e4891-3699-44ac-a18e-4d55e4f9eded service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:19.728015 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e4891-3699-44ac-a18e-4d55e4f9eded service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:19.737831 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e4891-3699-44ac-a18e-4d55e4f9eded service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:55:19.739627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e4891-3699-44ac-a18e-4d55e4f9eded service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:55:19.748796 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1470/2915] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:55:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:55:29.675386 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=3 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:55:29.745197 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=29 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:55:52.217032 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7b1db-2332-46a6-97dd-3f0dad023742 None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:55:52.274452 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7b1db-2332-46a6-97dd-3f0dad023742 None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:55:52.278854 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9f79c-e816-4535-b610-adcdd3b8d9c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:55:52.279531 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9f79c-e816-4535-b610-adcdd3b8d9c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:52.279814 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9f79c-e816-4535-b610-adcdd3b8d9c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:52.280089 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9f79c-e816-4535-b610-adcdd3b8d9c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:52.290306 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9f79c-e816-4535-b610-adcdd3b8d9c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:55:52.291771 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9f79c-e816-4535-b610-adcdd3b8d9c4 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:55:52.301586 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1446/2916] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:55:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:55:55.611000 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e4891-3699-44ac-a18e-4d55e4f9eded service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:55:55.613425 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-236e4891-3699-44ac-a18e-4d55e4f9eded service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:55:55.743322 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e4891-3699-44ac-a18e-4d55e4f9eded service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:55:55.749297 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f5653-f747-4a1f-8aad-c55ae5f506d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:55:55.750126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f5653-f747-4a1f-8aad-c55ae5f506d2 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:55:55.750508 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f5653-f747-4a1f-8aad-c55ae5f506d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:55:55.751151 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f5653-f747-4a1f-8aad-c55ae5f506d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:55:55.753304 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-869f5653-f747-4a1f-8aad-c55ae5f506d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:55:55.769006 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f5653-f747-4a1f-8aad-c55ae5f506d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:55:55.771625 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f5653-f747-4a1f-8aad-c55ae5f506d2 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:55:55.785239 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1471/2917] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:55:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:56:02.297471 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=26 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:56:05.779523 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=34 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:56:31.845312 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9f79c-e816-4535-b610-adcdd3b8d9c4 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:56:31.857503 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9f79c-e816-4535-b610-adcdd3b8d9c4 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:56:31.864035 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd05c8-340f-43bc-afbd-509849de4093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:56:31.864707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd05c8-340f-43bc-afbd-509849de4093 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:56:31.864986 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd05c8-340f-43bc-afbd-509849de4093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:56:31.865267 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd05c8-340f-43bc-afbd-509849de4093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:56:31.867491 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-26cd05c8-340f-43bc-afbd-509849de4093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:56:31.878260 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd05c8-340f-43bc-afbd-509849de4093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:56:31.879947 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd05c8-340f-43bc-afbd-509849de4093 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:56:31.888363 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1447/2918] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:56:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:56:41.884968 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:56:58.887883 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f5653-f747-4a1f-8aad-c55ae5f506d2 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:56:58.889692 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-869f5653-f747-4a1f-8aad-c55ae5f506d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:56:58.902008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f5653-f747-4a1f-8aad-c55ae5f506d2 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:56:58.906016 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80f8c7-a757-4307-a289-56878c4b8106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:56:58.906655 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80f8c7-a757-4307-a289-56878c4b8106 service neutron] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:56:58.906968 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80f8c7-a757-4307-a289-56878c4b8106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:56:58.907243 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80f8c7-a757-4307-a289-56878c4b8106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:56:58.909100 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-7b80f8c7-a757-4307-a289-56878c4b8106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:56:58.922823 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80f8c7-a757-4307-a289-56878c4b8106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:56:58.924244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80f8c7-a757-4307-a289-56878c4b8106 service neutron] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:56:58.935340 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1472/2919] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:56:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 08:57:07.338853 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd05c8-340f-43bc-afbd-509849de4093 service neutron] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:57:07.349030 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd05c8-340f-43bc-afbd-509849de4093 service neutron] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:57:07.353851 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489cd26-b512-4620-9968-ea5606f32cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49a1439029fa4b719bc114c3f59e0050', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:57:07.354636 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489cd26-b512-4620-9968-ea5606f32cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:57:07.354923 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489cd26-b512-4620-9968-ea5606f32cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:57:07.355200 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489cd26-b512-4620-9968-ea5606f32cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:57:07.357390 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-6489cd26-b512-4620-9968-ea5606f32cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:57:07.372095 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489cd26-b512-4620-9968-ea5606f32cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:57:07.373584 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489cd26-b512-4620-9968-ea5606f32cb8 service neutron] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:57:07.386541 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1448/2920] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:57:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:57:08.930368 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=6 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:57:17.380849 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=6 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:58:18.795040 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80f8c7-a757-4307-a289-56878c4b8106 service neutron] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:58:18.827783 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80f8c7-a757-4307-a289-56878c4b8106 service neutron] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:58:18.834241 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba520f-bd02-4777-8ce8-65f15570ef83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bd500f020f143339c5a4fc9f0728ee4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:58:18.835008 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba520f-bd02-4777-8ce8-65f15570ef83 service nova] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:58:18.835246 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba520f-bd02-4777-8ce8-65f15570ef83 service nova] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:58:18.835520 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba520f-bd02-4777-8ce8-65f15570ef83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:58:18.837953 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-d6ba520f-bd02-4777-8ce8-65f15570ef83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:58:18.861627 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba520f-bd02-4777-8ce8-65f15570ef83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:58:18.863260 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba520f-bd02-4777-8ce8-65f15570ef83 service nova] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:58:18.874154 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1473/2921] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:58:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:58:19.841833 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9874d-6741-4277-bad2-b1090a3aa08a None None] REQUEST_METHOD: `POST` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:58:19.842401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9874d-6741-4277-bad2-b1090a3aa08a None None] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:58:19.842401 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9874d-6741-4277-bad2-b1090a3aa08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:58:19.866561 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.auth.core [None req-8aa9874d-6741-4277-bad2-b1090a3aa08a None None] MFA Rules not processed for user `f320b56bf7ff4c36bbd9730e688612d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80130) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 08:58:19.867810 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-8aa9874d-6741-4277-bad2-b1090a3aa08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:58:19.875834 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1449/2922] 10.222.0.27 () {58 vars in 967 bytes} [Wed Jun 29 08:58:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 08:58:19.922430 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba520f-bd02-4777-8ce8-65f15570ef83 service nova] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:58:19.935238 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba520f-bd02-4777-8ce8-65f15570ef83 service nova] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:58:19.940765 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1efbf3-cc5b-4a44-89f5-06b91cd5a235 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b38411347754c058eb6640829078e1d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:58:19.941403 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1efbf3-cc5b-4a44-89f5-06b91cd5a235 service swift] REQUEST_METHOD: `GET` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:58:19.941684 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1efbf3-cc5b-4a44-89f5-06b91cd5a235 service swift] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:58:19.941957 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1efbf3-cc5b-4a44-89f5-06b91cd5a235 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:58:19.952341 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1efbf3-cc5b-4a44-89f5-06b91cd5a235 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:58:19.953951 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1efbf3-cc5b-4a44-89f5-06b91cd5a235 service swift] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:58:19.966215 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1474/2923] 10.222.0.27 () {60 vars in 1384 bytes} [Wed Jun 29 08:58:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 08:58:26.568640 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9874d-6741-4277-bad2-b1090a3aa08a None None] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:58:26.639572 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9874d-6741-4277-bad2-b1090a3aa08a None None] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:58:26.644365 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:58:26.645245 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:58:26.645528 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:58:26.645832 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] PATH_INFO: `/v3/users/340cc4d0b4c543fab577749f947598b1` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:58:26.647085 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=340cc4d0b4c543fab577749f947598b1)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:58:26.649004 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:58:26.700246 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '340cc4d0b4c543fab577749f947598b1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:26.700822 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '340cc4d0b4c543fab577749f947598b1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:26.712917 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '340cc4d0b4c543fab577749f947598b1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:26.713593 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '340cc4d0b4c543fab577749f947598b1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:26.724130 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '340cc4d0b4c543fab577749f947598b1'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:26.737291 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1450/2924] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:58:26 2022] DELETE /identity/v3/users/340cc4d0b4c543fab577749f947598b1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:58:26.942846 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1efbf3-cc5b-4a44-89f5-06b91cd5a235 service swift] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:58:26.955153 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1efbf3-cc5b-4a44-89f5-06b91cd5a235 service swift] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:58:27.011736 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:58:27.012603 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:58:27.012879 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:58:27.013148 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] PATH_INFO: `/v3/projects/50a457f985c4496ca41152bfa29ae9e3` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:58:27.014466 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50a457f985c4496ca41152bfa29ae9e3)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:58:27.016357 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:58:27.064570 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50a457f985c4496ca41152bfa29ae9e3', 'request_id': 'req-208c6118-6e5c-4b47-9795-61e9c6f45425'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.070554 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50a457f985c4496ca41152bfa29ae9e3', 'request_id': 'req-208c6118-6e5c-4b47-9795-61e9c6f45425'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.076991 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50a457f985c4496ca41152bfa29ae9e3', 'request_id': 'req-208c6118-6e5c-4b47-9795-61e9c6f45425'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.092622 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] The token cache is being invalidate because project 50a457f985c4496ca41152bfa29ae9e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:58:27.092908 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.093539 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.095132 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1475/2925] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:58:26 2022] DELETE /identity/v3/projects/50a457f985c4496ca41152bfa29ae9e3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:58:27.099272 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:58:27.100862 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:58:27.115537 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8854b-f714-4567-81b5-a3207a0d6e03 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:58:27.181783 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:58:27.182707 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:58:27.183000 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:58:27.183278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] PATH_INFO: `/v3/users/ce44ba2e39494236878484fe3f0576f0` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:58:27.184547 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce44ba2e39494236878484fe3f0576f0)` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:58:27.186719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:58:27.201254 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:58:27.242333 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce44ba2e39494236878484fe3f0576f0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.243088 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce44ba2e39494236878484fe3f0576f0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.255229 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce44ba2e39494236878484fe3f0576f0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.255957 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce44ba2e39494236878484fe3f0576f0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.265719 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.notifications [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce44ba2e39494236878484fe3f0576f0'} {{(pid=80130) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.279724 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1451/2926] 10.222.0.27 () {58 vars in 1223 bytes} [Wed Jun 29 08:58:27 2022] DELETE /identity/v3/users/ce44ba2e39494236878484fe3f0576f0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:58:27.473118 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] Authenticating user token {{(pid=80131) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:58:27.474714 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.fernet_utils [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80131) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:58:27.488786 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c6118-6e5c-4b47-9795-61e9c6f45425 admin admin] Validating token access rules against request {{(pid=80131) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:58:27.525374 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:58:27.555404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c145cd66ea9b4d3a85c89c24f585acff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b8a36f8c21e4c4f9394d7b2d795e615', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80131) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:58:27.556563 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:58:27.556848 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:58:27.557126 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] PATH_INFO: `/v3/projects/71a66c47fe674c598d64780f7bb261c1` {{(pid=80131) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:58:27.558446 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a66c47fe674c598d64780f7bb261c1)` {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:58:27.560404 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] RBAC: Authorization granted {{(pid=80131) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:58:27.612349 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a66c47fe674c598d64780f7bb261c1', 'request_id': 'req-d7aad8d0-acda-4c04-b6fb-87b8002fd543'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.619311 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a66c47fe674c598d64780f7bb261c1', 'request_id': 'req-d7aad8d0-acda-4c04-b6fb-87b8002fd543'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.625444 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a66c47fe674c598d64780f7bb261c1', 'request_id': 'req-d7aad8d0-acda-4c04-b6fb-87b8002fd543'} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.641961 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] The token cache is being invalidate because project 71a66c47fe674c598d64780f7bb261c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80131) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 08:58:27.642244 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.642918 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG keystone.notifications [None req-d7aad8d0-acda-4c04-b6fb-87b8002fd543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80131) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 08:58:27.644514 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: [pid: 80131|app: 0|req: 1476/2927] 10.222.0.27 () {58 vars in 1232 bytes} [Wed Jun 29 08:58:27 2022] DELETE /identity/v3/projects/71a66c47fe674c598d64780f7bb261c1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 08:58:37.276278 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=21,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:58:37.638984 dv-cinder-797608-37-iscsi devstack@keystone.service[80131]: DEBUG dbcounter [-] [80131] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80131) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 08:59:31.130020 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] Authenticating user token {{(pid=80130) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 08:59:31.131782 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:59:31.269049 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7fb7ab-b63d-4c3a-9f96-2812885a7da9 admin admin] Validating token access rules against request {{(pid=80130) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 08:59:31.337530 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b7b63-7ac0-420e-836b-82b6a8a8b217 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a08f31638d4eaab5ca83d6f493db6c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c7a300f406e1459583100a1086caee9f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80130) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 08:59:31.338187 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b7b63-7ac0-420e-836b-82b6a8a8b217 service placement] REQUEST_METHOD: `GET` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 08:59:31.338465 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b7b63-7ac0-420e-836b-82b6a8a8b217 service placement] SCRIPT_NAME: `/identity` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 08:59:31.338795 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b7b63-7ac0-420e-836b-82b6a8a8b217 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80130) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 08:59:31.341133 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.fernet_utils [None req-990b7b63-7ac0-420e-836b-82b6a8a8b217 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80130) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 08:59:31.353226 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b7b63-7ac0-420e-836b-82b6a8a8b217 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 08:59:31.355202 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b7b63-7ac0-420e-836b-82b6a8a8b217 service placement] RBAC: Authorization granted {{(pid=80130) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 08:59:31.368605 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: [pid: 80130|app: 0|req: 1452/2928] 10.222.0.27 () {60 vars in 1365 bytes} [Wed Jun 29 08:59:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 08:59:41.360934 dv-cinder-797608-37-iscsi devstack@keystone.service[80130]: DEBUG dbcounter [-] [80130] Writing DB stats keystone:SELECT=76 {{(pid=80130) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}